{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Non-linear Regression Demo\n",
    "\n",
    "_Source: 🤖[Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning) repository_\n",
    "\n",
    "> ☝Before moving on with this demo you might want to take a look at:\n",
    "> - 📗[Math behind the Linear Regression](https://github.com/trekhleb/homemade-machine-learning/tree/master/homemade/linear_regression)\n",
    "> - ⚙️[Linear Regression Source Code](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/linear_regression/linear_regression.py)\n",
    "\n",
    "**Polynomial regression** is a form of regression analysis in which the relationship between the independent variable `x` and the dependent variable `y` is modelled as an _n<sup>th<sup>_ degree polynomial in `x`. Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function `E(y|x)` is linear in the unknown parameters that are estimated from the data. For this reason, polynomial regression is considered to be a special case of multiple linear regression.\n",
    "\n",
    "> **Demo Project:** In this example we will train our model to imitate an artificial non-linear equation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To make debugging of linear_regression module easier we enable imported modules autoreloading feature.\n",
    "# By doing this you may change the code of linear_regression library and all these changes will be available here.\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "# Add project root folder to module loading paths.\n",
    "import sys\n",
    "sys.path.append('../..')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Dependencies\n",
    "\n",
    "- [pandas](https://pandas.pydata.org/) - library that we will use for loading and displaying the data in a table\n",
    "- [numpy](http://www.numpy.org/) - library that we will use for linear algebra operations\n",
    "- [matplotlib](https://matplotlib.org/) - library that we will use for plotting the data\n",
    "- [linear_regression](https://github.com/trekhleb/homemade-machine-learning/blob/master/src/linear_regression/linear_regression.py) - custom implementation of linear regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import 3rd party dependencies.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Import custom linear regression implementation.\n",
    "from homemade.linear_regression import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the Data\n",
    "\n",
    "In this demo we will use artificial non-linear data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "      <th>x</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>97.58776</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>97.76344</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>96.56705</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>92.52037</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>91.15097</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>95.21728</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>90.21355</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>89.29235</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>91.51479</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>89.60966</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          y     x\n",
       "0  97.58776   1.0\n",
       "1  97.76344   2.0\n",
       "2  96.56705   3.0\n",
       "3  92.52037   4.0\n",
       "4  91.15097   5.0\n",
       "5  95.21728   6.0\n",
       "6  90.21355   7.0\n",
       "7  89.29235   8.0\n",
       "8  91.51479   9.0\n",
       "9  89.60966  10.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the data.\n",
    "data = pd.read_csv('../../data/non-linear-regression-x-y.csv')\n",
    "\n",
    "# Fetch traingin set and labels.\n",
    "x = data['x'].values.reshape((data.shape[0], 1))\n",
    "y = data['y'].values.reshape((data.shape[0], 1))\n",
    "\n",
    "# Print the data table.\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Data\n",
    "\n",
    "Let's visualize the training and test datasets to see the shape of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8XHW5+PHPM5N93/ekSZo03TfSDUqhbAVEFkUuoAIXFK8i6nXF6/WnXl96RS8qKqIISN0KCCiL7JXKVtqmdF+TNvu+7/t8f3+cSUj3NJnJTGae9+vVV2fOnDnnOZ30yXee813EGINSSinfZfN0AEoppdxLE71SSvk4TfRKKeXjNNErpZSP00SvlFI+ThO9Ukr5uDMmehF5VEQaRGTvSV77iogYEUlwPhcR+YWIlIjIbhFZ6o6glVJKjd94WvSPAZcfv1FEMoHLgIoxm68A8p1/7gQenHyISimlJuOMid4Y8ybQcpKXfgZ8HRg74uoa4A/G8h4QIyKpLolUKaXUhARM5E0icg1QbYzZJSJjX0oHKsc8r3Juqz3d8RISEkx2dvZEQlFKKb+1ffv2JmNM4pn2O+tELyJhwH9hlW0mTETuxCrvkJWVRVFR0WQOp5RSfkdEysez30R63cwEcoBdIlIGZADvi0gKUA1kjtk3w7ntBMaYh4wxhcaYwsTEM/5CUkopNUFnneiNMXuMMUnGmGxjTDZWeWapMaYOeA64xdn7ZiXQbow5bdlGKaWUe42ne+UGYDNQICJVInLHaXZ/ETgKlAC/Az7nkiiVUkpN2Blr9MaYm87wevaYxwa4a/JhKaWUchUdGauUUj5OE71SSvk4TfRKKeXjJjRgSilvVNLQycYDDWQnhHPR7CQC7dqOUQo00Ssf8cjbpXz/hf2jz//3Iwu4aXmWByNSyntok0dNe2VN3dz78kHWFiTyzj0XER5k51Bdp6fDUspraIteTTu17b38/p0yqtt6qWvvo6Shi2C7jR99dCHJUSHkJIZztKnb02Eq5TU00atp55f/LOHxrRXMiA8nOSqYy+elcN3SdJKjQgDITYjg/YpWD0eplPfQRK+mjR+/fJD4iGCe21nDdUsyuO+GRSfdLzcxnOd319A3OExIoH2Ko1TK+2iiV9OCMYb175bRPTAMwL8tyzzlvrmJERgDZc3dzE6JmqoQlfJaejNWTQsdvUN0DwwTERzAwoxolmXHnnLf3IRwAI42ap1eKdAWvZomqtt6Abj3owu5dG4yxy14c4zcxJFE3zUlsSnl7bRFr6aFkUSfHhtKUMDpf2zDggLIiA1lW5nekFUKNNErL9bRN0hxvdUfvsaZ6NNiQsb13o+dk8m/Djdqf3ql0ESvvNgvNxZz5S/e4nB9JzVtvQQF2EgIDx7Xe29ZNYPQQDu//dcRN0eplPfTRK+81r6aDgaHDfc8vZvK1h7SokOw2U5dmx8rNjyIG5dn8tyumtGyj1L+ShO98lqH6ztJigzm/Yo2Xj/QQHps6Fm9/1Pn5wLw8FtH3RGeUtOGJnrllZq6+mnqGuD21TmkRIUwMOQgLfrsEn16TChXL0rj8a2V9Dr73yvljzTRK6902HkTdm5qFB9fYc1CmRZzdoke4MLZSfQODlPeon3qlf/SRK+8UnG91Qe+ICWSG5dnkR4TyjkzTj1I6lQyneWeyhat0yv/pQOmlFc6VN9JdGggSZHBiAjv3HPRhI6TGRcGQGVLjyvDU2paOWOLXkQeFZEGEdk7ZttPROSgiOwWkb+JSMyY174pIiUickhE1rkrcOXb9la3U5AcedoRsOMRHx5EaKCdipYePv7we7y4p9ZFESo1fYyndPMYcPlx214D5htjFgKHgW8CiMhc4EZgnvM9vxYRnT5QnZW69j52V7VzQUHipI8lImTGhfLm4UbeKWnmzcONLohQqenljIneGPMm0HLctleNMUPOp+8BGc7H1wCPG2P6jTGlQAmw3IXxKj/w6v46ANbNS3bJ8TJjw0YXItE+9cofueJm7O3AS87H6UDlmNeqnNuUGrdX9tUxMzGcvKRIlxxvpE4PmuiVf5pUoheRbwFDwJ8n8N47RaRIRIoaG/XrtLJsPtLMu0eauXJBqsuOOTbR17T14nAYHA7jsuMr5e0mnOhF5DbgKuDjxpiR/zXVwNgVITKc205gjHnIGFNojClMTJx8LVZNf+29g3zpiR3kxIfzHxfMdNlxR7pYJkUG0zfo4N5XDnLRfZtcdnylvN2EEr2IXA58HbjaGDO239pzwI0iEiwiOUA+sHXyYSp/8M+D9dR39POjjy4kPNh1PX8XZcaQFRfGJ1fOAGDDlgrKmnvo6h86wzuV8g3j6V65AdgMFIhIlYjcAfwKiAReE5GdIvIbAGPMPuBJYD/wMnCXMUbHnqtxeau4idiwQAonMDDqdJKjQnjz62tZOzsJgI4+K8E3dPS59DxKeaszNpuMMTedZPMjp9n/B8APJhOU8j/GGN4qbmJ1fuK4Z6g8WxnHTYpW39FPbmKEW86llDfRKRCUVzhU30ljZz/n5ye47RzRoYGEB30wrKNeW/TKT2iiV17h2Z01AG5N9CJCWkwoCRFBgCZ65T800SuPq2rt4ZG3S7l2cRqpZzkV8dm6Y3UOX798NuFBduo7+t16LqW8hU5qpjzuN/86gk3g65fPdvu5blxuTXn8m01HqO/UFr3yD9qiVx53oLaTxZkxE5pvfqKSooK1143yG5rolcdVtPSQNWb06lRIjgrR0o3yG5rolUf1DgzT2NnvoUTfxweDupXyXZrolUdVtVoDqzOnONEnRQbTP+Sgo1dHxyrfp4leeVSlhxL9yOCpn71+mIEhx5SeW6mppoleeVRFs5Xop7p0c/GcZG5ekcVj75bx9PtVU3pupaaaJnrlURUtvYQG2okPD5rS8wbabfzg2vlEhgSwv6ZjSs+t1FTTRK88qrLV6nEz2bVhJ0JEyE+KoLihc8rPrdRU0kSvPKqiuWfK6/Nj5SdFUtLQ5bHzKzUVNNErjznS2MWh+k6WZMV4LIb85AiaugZo6R7wWAxKuZsmeuUxf9lSQYBNuKEw88w7u8nMJGuaYm3VK1+miV55RP/QME9tr2Ld/BQSI4M9Fke+M9FrnV75Mk30yiNq2/po7x1kbUGSR+NIiw4lLMjOoTpN9Mp3aaJXHtHeOwhATGigR+Ow2YTz8hJ4dmcN3bqGrPJRmuiVR4wk+ugwzyZ6gM9eOJP23kE2bK3wdChKuYUmeuURo4newy16gKVZsSzPjuOJbZWeDkUpt9BErzzCmxI9wNy0KGrbdX565ZvOmOhF5FERaRCRvWO2xYnIayJS7Pw71rldROQXIlIiIrtFZKk7g1fTl7cl+sTIYLr6h+gdGPZ0KEq53Hha9I8Blx+37R5gozEmH9jofA5wBZDv/HMn8KBrwlS+pqN3kKAAGyGBdk+HAkBihNXFs6lLFyNRvueMid4Y8ybQctzma4D1zsfrgWvHbP+DsbwHxIhIqquCVb6jvXfQa1rzwGhf/kZN9MoHTbRGn2yMqXU+rgOSnY/TgbF3tKqc25Q6hrcl+gRni76xUxO98j2TvhlrrLXYzno9NhG5U0SKRKSosbFxsmGoacbbEv1Ii15LN8oXTTTR14+UZJx/Nzi3VwNjJy7JcG47gTHmIWNMoTGmMDExcYJhqOnK2xJ9fIQ1H7626JUvmmiifw641fn4VuDZMdtvcfa+WQm0jynxuNyhuk5++OIBbn10K7sq29x1GuUG3pboA+02YsMCtUWvfNJ4ulduADYDBSJSJSJ3AD8CLhWRYuAS53OAF4GjQAnwO+Bzbonaqaq1h8feKaOorIUvPL5Dh7BPI96W6MEq33hri94YM7rsolJnazy9bm4yxqQaYwKNMRnGmEeMMc3GmIuNMfnGmEuMMS3OfY0x5i5jzExjzAJjTJE7gz8/P5E937uMR25bRkVLDw9uOuLO0ykXGXYYOvuGiPKyRJ8Q4b2JftPhRtb85A1Km7o9HYqahgI8HcBkBAVYv6dW5sazbEYc7x5pAgo8G5Q6rSe3VbJ+cxngPYOlRiRGBrOjwrtKgH/YXEag3UZ9hzVqt6Klh5yEcM8GpaYdn5kCYUFGNPtqOhgadlDd1suFP3mDI426mIQ3McbwmzePsM+5GLe3JfqxLXqrM5lntfcO8sMXD/DgpiMU11s/y01e+o1DeTefSfQLM6LpH3JQ3NDFO8VNlDX3sPlIs6fDUmPsrmrnaOMHpQdvS/RJkcH0Dg6z7mdvMv87r/Ckhyc5+/uOavoGHVS09FBUbo1Z1JvFaiJ8JtEvSI8GYE9VO7urra/fujycd3nm/SqCAmyIWM8jgr2rcnjtknRuOzeb9NhQRIT3Sj3bUNiwtWL036i+w0rwmujVRPhMos+ODycyOIDd1W3sqbZKA7o8nHd5/UADF85K5N6PLAQgIzbUwxEdKzkqhO9ePY9Hb1vGgvRoyqewl8snHt7C/a8Xs6OilU+tL6KsqZuDdZ18Zk3u6C9GgKYuXcRcnT3valJNgs0mzE+P5r2jLVS0WP9BR+qayvOqWnuobuvlU+fncMOyTD68KI3QIO+Y0OxkshPCeG1/PQDF9Z186Ymd/PGOFcSFB7n8XOXN3bxd0kRb7wCtPQO8fqCeyBDrv+bFc5J5fncNh+u7CAuya4teTYjPtOgBrj8ng5KGLgaGHCxIj6ahs5/2nkFq2np587BOs+BJ28qsGvPynDgAr07yADPiw2nqGqCzb5C3ipvYV9PBnup2t5zr9QPWwPJDdZ1sL28F4G87qokKCaAgJZJFGTHYBM6ZEeu13T+Vd/OpRH/dknTOmRELWPVWgJLGTr73/D5u+/1Wqtt6PRmeX9ta2kJUSACzU6I8Hcq4ZMeHAVDe3EOJs/dWZYt7SjkbD9QjAoPD5phfJstz4rDbhLvW5nH/jUvIiA3V0o2aEJ9K9Dab8LMbFvPfH5rDJXOSANhS2sI/DzbgMPDHzeUejtB/bSltYVm2lbimgxnxVl/1suZujjS4L9F39g2ytbSFK+d/MJv3ylzrW8/It5/shHA+vCiNhIhgWrr7+fKTO3nsnVKXx6J8l08leoCs+DA+dX4umbFhpEWH8NNXDzM4bJidEsnj2yp0BSEP6Owb5GhjN0ud37amgxljWvRHnF1CK9yQ6LeVtTDkMNy8Imu0u+k9V8zhsrnJfGhh2jH7JkQE4zDwzPvV/OZfR3E4PN/XX00PPpfoR9hswu9uLSQk0M7c1Cj+68o5tPUMsvlok6dD8zsj3VxnJUd6OJLxCwsKICkymF2VbaM3QN2R6LeUthBoF5ZmxbIgPZqokAAWZUTz0C2FpMcc2ytpZM58gLqOPoqc9XylzsRnet2czLy0aF764vkE2IXYsCCCA2y8VdzERbOTz/xm5TIjiT4vKcLDkZydgpRINjlv4qfHhLqldLPlaAuLMmIIDbLz1XUF1Hf0IXLy8laCcyrlhIgguvqHeG5X9Wh5R6nT8dkW/YjMuDBSo0MJCbSzPCeOt4tP3aL3hmHvvqikoYugABuZXtZv/kw+e8FMBoYcAKydnUhH3xDtPYMuO353/xB7qttZ4azJL86MYd28lFPun+BcHOWi2UlcPCeZF/fUMTTscFk8ynf5fKIf6/z8BIobuqhr7+OBN0r48hM7j0nun/vz+3z1r7s8GKFvKm7oIjchnAD79PpxOzcvgYtmJxEaaOfcmQmAa8s3OyraGHYYlufEj2v/jNhQzs9P4MblWXx4YRot3QO8q9N8qHHw6dLN8VbnJQIHuX/jYf5aVMWQw3Dp3GSuWJCKMYbNR5uPqYMq1yhu6GRRRoynw5iQn9+4mIrmntHeQhUtPSzIiJ7w8Vq6B6hr72NGfBhlzdZN3lnJ4ytpBQfY+eMdKwDoGxwmMjiA53fVsGaWrtCmTm96NbEmaU5qJFctTGXD1krsNiE3IZz/fekgww5Dc/cAbT2DVLX2aAnHRQ7WdfClx3dQ1do77erzI6JCApmfHk12fDh2m3CormPCx3I4DBfft4krf/EWX9iwg9r2Xuw2ISky5KyPFRJo57J5Kby8t85t/fuV7/CrRC8i/Pj6hVwwK5GvXlbAnWtyqWjpobKlZ/SGYd+gQweluMjzu2r4+84ajIGCadTj5mRCg+zMSo5kxySWrKzt6KPVWeM/WNdJbVsfKVEhEx5bcMfqHETg2gfeoUYHA6rT8KtED1a3ufW3L+fTa3KZ6WxlljZ3UzxmpsvKVm0huUJpUzfpMaH84qYlXDp3+vd0WpwZza7Ktgl/4yt3lmqWZcdS295LeUsPqdFn35ofMTctisduX05z94BOya1Oy+8S/VjZI6Mfmz4Y/QhQ1aqtI1cobeqhICWSqxelTbsbsSezODOGjr6hCS/nNzIb5pr8RBzGmlI7NWZyPZHmpVlTSmjjRJ3O9P/fNwkJEUFEBgdQ2tRNcUMn+c4WvtY8J8/hMJQ1dfvUsneLM62RvTsnWL4pb+4hyG4b7fs+MOwgbRIterBu0CZHBWvjRJ2WXyd6ESE7IZzSpm5KGrpYmBFDfHgQVdo6mrT6zj56B4d9KtHnJUUQHmQ/6xGpQ8MOKlt6KG/uJiMu9Jh/k8mUbkZkxoZp40Sd1qQSvYj8p4jsE5G9IrJBREJEJEdEtohIiYg8ISKun8DbhbITwtle3kp9Rz8LM6LJiAujoqWHTYca+P4L+zXpT1Cpc36YXB9K9HabsHZ2Ev/YXUvf4JnnTGrtHqBvcJhH3i5l7f9tYltZKzPiwkiMDCY00JqmebKlG7D611e19tLQ2afTGKuTmnCiF5F04AtAoTFmPmAHbgTuBX5mjMkDWoE7XBGou+QkhNMzMEygXbhqYSoZsaG8U9LMbb/fxiNvl/LwWzpL4EQcddaxcxJ9J9ED3Lwii/beQV7cU3vGfT/64Lt8/i87eLKokiGHoamrnxnx4YgIWXHWpGlp0ZNP9JlxYdS293L7Y9v48pM7J3085XsmW7oJAEJFJAAIA2qBi4CnnK+vB66d5DncKifB+g93yZxk4iOCWZUbT3pMKD++fiGXzk3mhd01Osx8AkoauggJtJE8gT7i3mxVbjy5CeH8ZUvFaferbuvlaFM3rx+o50hjN7Fh1syUI7NiZjn/To2Z/L9PRmwoDgN7qzt0nWR1UhNO9MaYauD/gAqsBN8ObAfajDFDzt2qgPSTvV9E7hSRIhEpamz03OpPC9JjCLAJn1w5A4BPrJzBO/dcxA2FmXx0aTpNXQP86o0Stpa2eCzG6ebht47yh81lLMuOwzZN5p8fLxHhpuVZFJW3crj+1GsSFzlX1AoNtBMUYOPnNy7BJjDfuYh9QXIk0aGBxLtgacLM2LDRx3UdfeMqKyn/MpnSTSxwDZADpAHhwOXjfb8x5iFjTKExpjAx0XNDuPOSItj93cs4Ny/hhNcuLEgiMiSAn79ezG2/30pDZ58HIpxe2noGuPflg1xYkMRvPnGOp8Nxi4+ek0GQ3cZDbx7lH7trGRp28PWndnHP07tH9ykqayUiOIBHbivkJ85Beju+fRnLsq0eN59bO5MX7l59ypkqz0bGmERvjHYPVieaTOnmEqDUGNNojBkEngHOA2KcpRyADKB6kjG6XVjQyaf8CQm08+RnVvGbT5zDwJCDX24smeLIpp9/7KllcNjw5UtnER7sm1MpxYUHccWCFJ7aXsVdf3mfDVsreHZnDU+/X0VbjzWqeltZC0uyYjh3ZgLXLLa+1EY7yzdg/cxlxoWd9PhnKzUmhKAAG7NTrNHH2gNHHW8yib4CWCkiYWI1Sy4G9gNvANc797kVeHZyIXrWnNQoLp+fwr8ty2TD1gr2VLXjcBidD+cUnt1Rw8zE8NGBPL7qPy+ZxadW55AaHcL9G4vpH3IwOGz4xcYSbnroPQ7WdY6uX+xugXYbGz69kl/etARwzwIpanqbTI1+C9ZN1/eBPc5jPQR8A/iyiJQA8cAjLojT4762roCEiGDuWL+Ned95hT/o+rMnaO7qZ2tZC1cvSndJScKbZSeE899XzeXy+Sk0dQ0QaBfSY0J59J1SShq7uHXVDG5enjVl8ZwzI5a8pAhCA+2a6NUJJtXrxhjzHWPMbGPMfGPMJ40x/caYo8aY5caYPGPMx4wxPtGxNyYsiJ/+2yK6+oew24TndtV4OiSvM5Jg5qf7dmt+rMvmWguFFM6I4841ucyID+OJO1fyvWvmkxQ1tT2ORrptaqJXx/PrkbFn69yZCez73jr+/bxsdlS0unS1IV9Q7ZxBMc0Fg4Cmi2XZsSzKiOYjS9O59dxsNn31QnITPTclc2acjpJVJ9JEf5ZEhAsLrEmp3i7RhcbHqnb29kifZksGTkaA3cazn1/NxwozATxeshpZ0KR/SLtYqg9oop+ARRkxRIUE8Nr+OgDtt+xU3dZLZEgAUSGBZ95ZucXqvAT6Bh28W6LTFqsPaKKfgAC7jevPyeTvO2u48w9FLPzuq2wv1wFVNW29pPtR2cYbnZsXT2RwAC/tPfMUDcp/aKKfoK+tK2BmYjiv7q9HBL79931+P1VCVasmek8LDrBz8ZwkXttf7/c/j+oDmugnKDTIzvrbl/O7Wwr56Q2L2V/bwd0bdowOmPFH1W29flWf91aXz0+htWeQbWVnN52y8l2a6CchIzaMS+cmc+WCFL5+eQGv7a/nx68c8nRYHtHRN0hn35C26L3A6vxEAu3CG4caPB2K8hKa6F1ARPjchXmszI1nX3W7p8OZckcbu3jgDWt6CG3Re15EcAArcuL550FN9Mqiid6F8pIiKG7o8rvpER5+u5Tf/usocOwEW8pz1s5OoqShS/vUK0ATvUvlJ0fQMzBMTXsfzV39fPmJncd8fd5f08HF922ipdu36vgVzT3kJITzP9fMY6FzGl7lWWsLrBlhNx323BTgyntooneh/CRr9sDi+k5+vekIz+yo5t9/v40NW61FKjYfbeZIYzd7fay8U9HSw/z0aG5Zle1z889PVzkJ4SRGBo/Oi6/8myZ6F8pPsoa+v3ukmT+9V861i9OYkxrFk0WVAJQ3W8vr+cpcJMYYBocdVLf1MsNFU+4q1xARCmfEUqQ9bxSa6F0qNjyIhIggHn7rKMMOw39eOosPLUhhR0Ub9R19lDVbCd4XEv3QsIPV977BvS8dZNhhRtdAVd7jnBmxVLf1UteuC+b4O030LpaXFIHDwHeunseM+HDWzbNmN3x1f/0HLfrm6Z/ojzR2U93Wy5+2WNM1j6yBqrxHoXM1qyIdte33NNG72B2rc/nG5bP5xAprLvK8pAhyE8J5cXft6BJv5Sdp0Xf3D52wzZvtrmoDoG/QGn05QxO915mXFkVIoE3LN0oTvatdOjeZz144c3QWQxHhsnkpbD7azLDDEBUSQGVLzzFdMA/UdrDoe6+yq7LNU2GPy/byVl52zqGyZ8wN5aAAG8mRUzv3ujqzQLuN5TnxbDrU4HddftWxNNFPgcvmJY8+Xp2fQFf/0DFdLDceqGfIYdhX0+GJ8Mbt568f5lt/2wtYiX5RZgzBATYyY0O1t42XunROEmXNPRxp7PZ0KMqDNNFPgcUZMSRFBgNwfr7Vv3nsDdl3nFPKVrd5d+2+uL6L5u4B6tr72F/TwbIZsVy7OJ0LZiV5OjR1ChfNsRoZrx+o93AkypM00U8Bm024amEaCRHBowtG//S1wxxp7KJvcJjtFVYNdaSG743aewep67B6bzy3q5r+IQcLMqK59/qF/L8Pz/VwdOpU0mNCmZsaxT8P6HQI/kwT/RT5+uUFvPiF1cxMjODjK7LYXt7KD/5xgKKyVgaGHAQF2EZXaBrRNzjMbb/f6hUDrEoaukYfr3/X6mmzIifeU+Gos7A8J469Ne1ap/djAZ4OwF+EBNoJCbQD8IPrFuAw8MLuGnISwgm0CxfPTmJHxbE3Y8ube9h0qJHCGbHM9/DUAsX1nQAE2ITqtl7ykyJIidYbsNPBzCRrao66jj5So3XSOX80qRa9iMSIyFMiclBEDojIKhGJE5HXRKTY+Xesq4L1JSty4ujsG2LD1gpWzUxgVnIk9Z19DAw56Oofoqypm8bOfsA7BlgVN3QREmhjqbP0NHKvQXm/mYnhABxp0Buy/mqypZv7gZeNMbOBRcAB4B5gozEmH9jofK6OszzHGszSMzDMpXOTyYgNxRiobe/lFxuLufbX79DQadXEK1s8X7s/XN9JXlIEc1OjADg/P8HDEanxyku0puY40th1hj2Vr5pwoheRaGAN8AiAMWbAGNMGXAOsd+62Hrh2skH6orSYUDLjrK/Rl85JHp3Hvaq1l12VbbT1DHKoziqXeKpF39DRx1ee3MWDm46wtbSFualRXFiQyOyUSFbkxnkkJnX2EiODiQwO0ETvxyZTo88BGoHfi8giYDvwRSDZGDOyMnEdkHyyN4vIncCdAFlZWZMIY/q6bnE6h+u7SIkOYdC5vmdVaw8HnQl+h3MAVW17L4PDDgLtU3vv/PUDDTz9fhUACzOi+eq6ApIiQ7iwQLtTTiciQm5ShCZ6PzaZRB8ALAXuNsZsEZH7Oa5MY4wxInLSW/3GmIeAhwAKCwv9sjvAly8rGH2cEh1CgE14q7iJ9t5BAPZUWb1tHAZq2/rIig/jnwfr+elrh7lifip3rM4ZvcHrDmXN3QTZbTx2+zKWZMYSGuS+cyn3mpkYzrvO8RrK/0ymiVgFVBljtjifP4WV+OtFJBXA+bd24B2HQLuNVTPj+cee2tFtvYPDBDhHnFa09PBkUSW3P1ZEXXs/P3nlEL9+o4SntlfxkV+/Q32H62coLGvqJis+jHNnJmiSn+bykiKo6+hjp5dPs6HcY8KJ3hhTB1SKyEiz9GJgP/AccKtz263As5OK0I9cuSCVka7OQc4yzbw06+bnW8WNfOfZfZw7M563v7GWwhmxvFncxONbK3i/oo1PPrKFvsFhl8ZT1txNdny4S4+pPOP6pRlkxoXyyUe26PKCfmiyRd+7gT+LyG5gMfBD4EfApSJSDFzifK7G4bK5ydjEGs04MhvkgoxoAu3Cb988SlCAjZ/esJiQQDsrc+PZU93Ozso2CpIjOVzfNe5J0UZ685xMZ98gww6Dw2Eob+4hJ0FnpfSz6GEuAAAXe0lEQVQFSVEhPHLrMjr7hnizWJcX9DeTSvTGmJ3GmEJjzEJjzLXGmFZjTLMx5mJjTL4x5hJjjE6GPU7xEcFcMT+VNbMSR3vhpEaHcsGsRM7Li+ev/7FqdJDSytx4hh2GIYfh9tXZgDXA6kxe21/Pyh9upLTpxD7Vrd0DrL73DX7++mHqOvroH3IwQ1v0PiM/KYKI4AAOO2/2K/+hUyB4mQc+vpT//cgC0mOsRJ8YEczDty7jz59ayazkyNH9zpkRS6BdCAm0cfWidAJsQmnzB8m7tKmb+18vxuE49j73K/vqcBjYV3PitAqPvVtGe+8gj71bNjqTZk6CJnpfISLMSo4Y7dWl/Icmei810qJPjAo+6euhQXZW5yVw0ewkQoPsZMWFja5gBfD9F/bzs9cPs6Pyg0UnjDH867D1tX3s3DVgLXzy2LtlzEqOoLNviPtePQTogiK+piAlkkP1nTrvjZ/RRO+lRm6CjrTsT+ahWwq5/8YlgJWQS5us0s3e6nb+edDq7PTinjoA3i1p4rdvHh2dVuH4RL+1rIX23kG+fdVcLixI5GBdJ2FBdp0bxccUJEfS1jM4+nOg/INOaual1s1LYcOnjy3XHG/sAKrshHC2lLZgjOH375QRGRLA3NQoXtpTyxcuzuczf9pOZ5+1XOH89ChKGrqo7+jjzcONpMeGUlJvJf75adH87pZCdle1Y7cJdl1QxKfMSrF+ng7WdZIUpZPS+QtN9F7KbhNWzRz/NMA5CeH0DAzT2NnPtrIWVuclcMmcZL7y11185o9FdPYN8bV1BYQH2alu62X95nI+/Mu3aejsJz48iLWzk0iICCY2PAhgdN585VsKkkcSfQdrZunEdP5CSzc+YqR3zPbyVipaeliSFcOVC1I5Ly+e9462cPHsJO5am8dt5+WQlxTBwJCDhs5+rl6URnP3AG8cbGBWcoSHr0K5W3xEMAXJkfxjd+2Zd1Y+QxO9j8h19o7543vWoiBLsqwpC/54+woeuNnqyTMiL8lK6AszovnCxfkANHcPnLZMpHzHzSuy2FXVPjrFhvJ9muh9REZsKIszY3j3SDN2mzA/zVqoxGYTPrQw9Zh67OyUKOakRvGVywqYmRhOnLNck68ter9w3dJ0QgPt/GVruadDUVNEE72PEBH+44JcAOakRp52bprw4ABe+uL5XDArERGh0FmP1xa9f4gKCWTt7ETeKm7ydChqimii9yGXzk1haVYMl81NOav3rZmVSGigXRO9H1maFUtVa+9pp8NQvkN73fgQu0145nPnnfX7bl6exWXzkokODXRDVMobLcmKAWBnRRuXzTu7hoGafrRFr7DZhKRI7VPtT+alWZPl7dBpi/2CJnql/FBIoJ25qVHsqGg9885q2tNEr5SfOmdGHEVlrTzydqmnQ1FupoleKT9190V5XDArke+/sF9b9j5OE71Sfio2PIgfX78QgM1HdT1ZX6aJXik/Fh8RTH5SBFuO6vpAvkwTvVJ+bnlOHNvLWxkadng6FOUmmuiV8nPLc+Lo6h/iQK2uPOWrNNEr5ecKs+MA2Fmlfep91aQTvYjYRWSHiLzgfJ4jIltEpEREnhCRoMmHqZRyl5SoEGwCDR06HYKvckWL/ovAgTHP7wV+ZozJA1qBO1xwDqWUm9htQlx4EE1durygr5pUoheRDOBDwMPO5wJcBDzl3GU9cO1kzqGUcr+EiGAaOwc8HYZyk8m26H8OfB0YuV0fD7QZY4acz6uA9EmeQynlZomRwTRqi95nTTjRi8hVQIMxZvsE33+niBSJSFFjY+NEw1BKuUBCRDBNnf0UlbXw/K4aT4ejXGwy0xSfB1wtIlcCIUAUcD8QIyIBzlZ9BlB9sjcbYx4CHgIoLCw0k4hDKTVJIy36X71Rwt7qdj68KM3TISkXmnCL3hjzTWNMhjEmG7gR+Kcx5uPAG8D1zt1uBZ6ddJRKKbdKiAhiYMjBrso2mroG6Bsc9nRIyoXc0Y/+G8CXRaQEq2b/iBvOoZRyocTIYABaewYBqGrt9WQ4ysVcssKUMWYTsMn5+Ciw3BXHVUpNjYSI4GOeV7f1kpeki8X7Ch0Zq5Q6IdFXtfZ4KBLlDrpmrFJqtHRjE7CJaOnGx2iiV0oRGxaETSArLgyD1uh9jSZ6pRR2mxAfEUxeUgS9g8NauvExWqNXSgHw/WvmcfdF+WTEhFGtLXqfoi16pRQAl89PBeDNw400dPbTOzBMaJDdw1EpV9BEr5Q6xqLMGAA2HWpga1kLV8xPZXlOnIejUpOhiV4pdYxzZ8aTGBnM/3tuH42d/TgcRhP9NKc1eqXUMQLsNq5dnEZjpzWbZUOnzmo53WmiV0qd4N+WZRIaaCcuPIh6XXlq2tNEr5Q6QV5SJPu+t441+QnaovcBmuiVUidlswlJUSE0dPZjjM4kPp1poldKnVJSZDADQw46eofOvLPyWprolVKnNDIHTkOn1umnM030SqlTSooMAbTnzXSniV4pdUrJUdqi9wWa6JVSp5QUZbXo6zu0RT+daaJXSp1SRHAAYUF2GjTRT2ua6JVSp5UUGcyuqja2lrZ4OhQ1QZrolVKnlZ0QzvbyVj7x8BZ6BrSb5XSkiV4pdVq/uGkJP7xuAQPDDnZWtHk6HDUBE070IpIpIm+IyH4R2SciX3RujxOR10Sk2Pl3rOvCVUpNtaiQQK5alIoIbC3T8s10NJkW/RDwFWPMXGAlcJeIzAXuATYaY/KBjc7nSqlpLCokkDkpUVqnn6YmnOiNMbXGmPedjzuBA0A6cA2w3rnbeuDayQaplPK85Tlx7KhoY3DY4elQ1FlySY1eRLKBJcAWINkYU+t8qQ5IdsU5lFKetWpmPL2Dw/x9R7WnQ1FnadKJXkQigKeBLxljOsa+Zqwp70467Z2I3CkiRSJS1NjYONkwlFJudsmcZJZlx/I/z++nps1aPHxgyMHOyjaGHTq7pTebVKIXkUCsJP9nY8wzzs31IpLqfD0VaDjZe40xDxljCo0xhYmJiZMJQyk1Bew24b6PLWbQ4eBHLx0EYP27ZVz7wDus+fEbHK7v9HCE6lQm0+tGgEeAA8aYn4556TngVufjW4FnJx6eUsqbZMWH8anVuTy3q4bdVW28dqCezLhQegaG+K9n9ui89V5qMi3684BPAheJyE7nnyuBHwGXikgxcInzuVLKR3zmglziw4P49rP72F7eytWL0vjmFXMoKm/l15uOaLL3QgETfaMx5m1ATvHyxRM9rlLKu0WGBPKlS/L59rP7AFhbkMTSrFjeONTAT145REffIN+8Ys4J7+vuH2J7eStrZmmpdqrpyFil1Fm7cXkWuQnhRIcGsjgzBptNeODmpVw2N5knt1XiGHNz9r//vocNWyv443vl3PLoVuradcrjqTbhFr1Syn8F2m08dEshrT0DBNit9qLNJqybl8Kr++v589YKHnyjhPtuWMyf3qtgUUY0qdGhAJQ2dZMSHeLJ8P2OJnql1ITkJUWcsO28vAQAvv/8fgaGHdy94X0A9tV0UONsyZc3d7NqZvy4z3O4vpO0mFAigjVdTZSWbpRSLpMSHUJuYjgDww7Cg+w0dQ0QaBeGHIZG53KE5S094z7esMNw3QPvcP/rh90Vsl/QRK+Ucqk1+YkE2W388uYlANyyKnv0NRGrRQ/w2v56Nh6oP+2xmrv66R4YnvQcO8/vqmFnpf/OvKmJXinlUl++bBbP3X0eF81O5unPruJr6wrITQhHBJbNiKO82WrR/88L+0YHXjV09HH3hh28X9F6zLFqneWefTUd9A4Mj+v8fy2q5Jn3q0aft/cO8pUnd/Hbfx0Z9zVsOdrMk9sqx72/t9Oil1LKpaJCAolKCQTgnBlxAFw6N5ldVW3MToniqe1V1Hf0UdnSi02gpq2Xm3/3HmXNPeyoaOXlL60ZrcfXtltTLQw5DLuq2liWHcdbxY30DAwz7DAsz4kjKTKYzv4hokIC6R8a5vsv7Cc9NoyPLM0A4OW9tQwMO2jqGv9yiA9sOsI7JU2cmxdPRmyYK/95PEITvVLK7b55pdWv/tG3S+nqH+KVfXUAOAz8+OWDlDX38K0r5/DDlw5w8X2b+MjSDL5x+ezRFj3A9vJW6jv6+OLjO0e3rciJ40MLU/l/z+4jLymCqxam0tE3hKOlB2MMIsLfnJOwNXUNjDveQ3UdDDsM698t41sfmuuKfwKP0tKNUmrKzIi3WsdPFlVicw63fH53LbkJ4Xx6TS6/umkpKVEhPPzWUfoGh6lr7yPIbiM3MZz3jjbz8t46kiKDeflL53P3RXlsKW3hJ68coiA5kq6+IX7+ejEAXf1DtPUMUt7czZbSFoICbDR1HtuiP9rYRVf/iUsjtvUMUN/RT3CAjQ1bK8ddMhrxTkkTDR3eNVZAE71SasosyowhKiSAvdUdrMiJJzI4gGGHGR0t+6GFqXzh4nwGhw07K9uobe8jJTqEqxel8VZxExsPNnDJ3GRmp0TxqfNzCQuy09k3xLevmsuDn1hKoF3ITQgHoKKlh9+9dZRAm42blmXS2T9Ee+8ge6raaeke4Ir73+Jjv9lMe8/gMTEerLMmZ/voORl09Q9x6DSTtd3x2DZ++toHPYKGhh38+++38eBZ3A+YCprolVJTJiEimL98eiVJkcFcsSCFOWlRAJyfnzC6T+GMOGvZwtIW6tr7SI0O4dZV2YQG2hkYcnDpXGuJi+jQQO5am8cV81M4Ly+eJVmx/Otra7n/Rqu3z66qNv5aVMVHlqYzJ9U6z683lfDhX73Nd5/bR/+Qg+L6Tr74xI7Rc3f1D3Gw1ppt/drF6YBVxgHoHxpmYOiDRVdq2nrZeLCBx7dWjI4EbuzqZ2DYwdHG7pNe/8G6DnYcd8N5KmiNXik1peanR/PeNy9GBCpbethZ2caK3A8GUEWHBTLbuWxhbUcv52TFEhsexC3nzuDp7VWcO2aw1V1r8445dlpMKFGh1o3gh948Sv+Qg9tX51Dh7Omz+UgzAM/tqmFeWhTXLk7nBy8eYFtZC1lxYVz0f5uwiRATFkjhjFjCguwcqLVa9J9aX0RIoJ37bljEHzeXExxgtZMbOvvZU93OosyY0XsKpU3dlDZ1s728levPsW4KP1lUybf+tof48GDe+6+pnQ5MW/RKqSlnswkiwufX5vP3z513wqjXFTlxFJW3UNnSS4pz6oSvr5vNpq+tJTjAftpjRwQHEBsWSFVrL9nxYcxKjiQhMhiwumnanTcHPro0g0+snEFCRDA/e+0wr+2vp3tgmM7+IQqSI7HZhFnJkRyq6+RgXQdvFTfx3tFm/rG7lp+8coh7Xz5IYmQwNoHXneMBRubxqWrt4f7XD/PVv+7iSGMXAD9++RCDw4a6jr6T3htwJ030SimPiQ4LZK6zfDPWhxel0jdolUlSoqwkbbfJuKdByIqzbvpeWJAEQHx4EGCNtL1ifgr337iYj6/MIjTIzmcvnMm7R5p5cNMRsuLCuHlFFjcUZgIwOyWSg3Ud/Om9cgA6+4Z4bmcNAIPD1rEKs+N45v1qGjr7Rlv0DgOv7LOS/5NFlXT2DdLU1c+ijGgAyppOXtpxF030Simvc86MOD534UwAMuPOvh97hvM9F822En2is0UPMDMxgmsWp49+M/j4iiySIoOpbuvl0rnJ/PC6BXzUWW6ZnRJJa88gT26rGq3zbz7azKrceP79vGxuWZXN19YV0NozwM2/20LlmOkdegeHCbAJT2+vorjBatWvdcZTqoleKaXga+sK+McXVrPW2So/G3NTo4gLD2J5jjVgKyTQPvptIMfZK2dESKB99JfKunkpx7w2ktxzE8N5+NZCApxlnxW5cXznw/PIS4pgWXYcP7huPiUNXby2v56EiKDR9992bjZNXQM8td0aqXuBs3eRtuiVUgoQEealRWOznWp9o1O7c00ub3zlQkICP6jnjyTg7OMSPVjz8Tx713mjvxhGLM+J41c3L+GJO1eRHhPKrORIABZnxhyz38isndVtveQnRZIQEUSgXbhzTS5gzbUDMDslitToEEqd8/08v6uGEmdr35000SulfE6g3UZ0WOAx2xIirPJNTvyJid5mExYdl7zB+mVz1cK00WMtSLdq7Mcn+qTIEDLjrJvGqdEhzE6JYklmLElRIcxKjqCzb4jU6BBCg+xkx4dT2tRN3+Aw//nEztHWvjtp90qllF9IjAwmNizwhF8AZ+PTa3JYkhVDTFjQCa8tzYp19hIKGZ3yAaz7DYfru0ZLRtkJ4by8t5a91e0MOQxLs078BeNq2qJXSvmFz1wwkx9ct2BSx8hLiuTG5VknfW1pVixgtegTI4NHbwAXzrC2j5SMZiaG09ozyEt7rfl+ljpfdydN9Eopv7A4M4YrF6S67firZsZjE8h31vFHLMu26v55idaKXCNdPv+wuYysuLDRkpI7uS3Ri8jlInJIREpE5B53nUcppbzBrORItvzXJazMPXaZxKz4MDZ8eiU3Lrf65uclRbAkK4bBYcOSKSjbgJsSvYjYgQeAK4C5wE0iMv3n+lRKqdMY219/rFUz4wkL+uCW6MfOsZL+SLnH3dx1M3Y5UGKMOQogIo8D1wD73XQ+pZSaNq5dksaRxi6uWui+UtJY7kr06cDYdbiqgBVuOpdSSk0rYUEBfPuqqStyeOxmrIjcKSJFIlLU2NjoqTCUUsrnuSvRVwOZY55nOLeNMsY8ZIwpNMYUJiYmuikMpZRS7kr024B8EckRkSDgRuA5N51LKaXUabilRm+MGRKRzwOvAHbgUWPMPnecSyml1Om5bQoEY8yLwIvuOr5SSqnx0ZGxSinl4zTRK6WUj9NEr5RSPk6MMZ6OARFpBMon8NYEoMnF4Xg7vWb/4Y/Xrdd8dmYYY87YP90rEv1EiUiRMabQ03FMJb1m/+GP163X7B5aulFKKR+niV4ppXzcdE/0D3k6AA/Qa/Yf/njdes1uMK1r9Eoppc5surfolVJKncG0TfT+slShiJSJyB4R2SkiRc5tcSLymogUO/+emmVq3EREHhWRBhHZO2bbSa9RLL9wfu67RWSp5yKfuFNc83dFpNr5We8UkSvHvPZN5zUfEpF1nol6ckQkU0TeEJH9IrJPRL7o3O6zn/VprnlqP2tjzLT7gzVR2hEgFwgCdgFzPR2Xm661DEg4btuPgXucj+8B7vV0nJO8xjXAUmDvma4RuBJ4CRBgJbDF0/G78Jq/C3z1JPvOdf6MBwM5zp99u6evYQLXnAosdT6OBA47r81nP+vTXPOUftbTtUU/ulShMWYAGFmq0F9cA6x3Pl4PXOvBWCbNGPMm0HLc5lNd4zXAH4zlPSBGRKZmPTYXOsU1n8o1wOPGmH5jTClQgvV/YFoxxtQaY953Pu4EDmCtRuezn/VprvlU3PJZT9dEf7KlCk/3jzedGeBVEdkuInc6tyUbY2qdj+uAZM+E5lanukZf/+w/7yxTPDqmJOdz1ywi2cASYAt+8lkfd80whZ/1dE30/mS1MWYpcAVwl4isGfuisb7v+XTXKX+4RqcHgZnAYqAWuM+z4biHiEQATwNfMsZ0jH3NVz/rk1zzlH7W0zXRn3GpQl9hjKl2/t0A/A3ra1z9yFdY598NnovQbU51jT772Rtj6o0xw8YYB/A7PvjK7jPXLCKBWAnvz8aYZ5ybffqzPtk1T/VnPV0TvV8sVSgi4SISOfIYuAzYi3Wttzp3uxV41jMRutWprvE54BZnj4yVQPuYr/3T2nH15+uwPmuwrvlGEQkWkRwgH9g61fFNlogI8AhwwBjz0zEv+exnfaprnvLP2tN3pSdxN/tKrDvYR4BveToeN11jLtYd+F3AvpHrBOKBjUAx8DoQ5+lYJ3mdG7C+vg5i1STvONU1YvXAeMD5ue8BCj0dvwuv+Y/Oa9rt/A+fOmb/bzmv+RBwhafjn+A1r8Yqy+wGdjr/XOnLn/VprnlKP2sdGauUUj5uupZulFJKjZMmeqWU8nGa6JVSysdpoldKKR+niV4ppXycJnqllPJxmuiVUsrHaaJXSikf9/8BlfyrZ5r5j0cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Init and Train Linear Regression Model\n",
    "\n",
    "> ☝🏻This is the place where you might want to play with model configuration.\n",
    "\n",
    "- `polynomial_degree` - this parameter will allow you to add additional polynomial features of certain degree. More features - more curved the line will be.\n",
    "- `num_iterations` - this is the number of iterations that gradient descent algorithm will use to find the minimum of a cost function. Low numbers may prevent gradient descent from reaching the minimum. High numbers will make the algorithm work longer without improving its accuracy.\n",
    "- `learning_rate` - this is the size of the gradient descent step. Small learning step will make algorithm work longer and will probably require more iterations to reach the minimum of the cost function. Big learning steps may couse missing the minimum and growth of the cost function value with new iterations.\n",
    "- `regularization_param` - parameter that will fight overfitting. The higher the parameter, the simplier is the model will be.\n",
    "- `polynomial_degree` - the degree of additional polynomial features (`x1^2 * x2, x1^2 * x2^2, ...`). This will allow you to curve the predictions.\n",
    "- `sinusoid_degree` - the degree of sinusoid parameter multipliers of additional features (`sin(x), sin(2*x), ...`). This will allow you to curve the predictions by adding sinusoidal component to the prediction curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial cost: 137723281.24\n",
      "Optimized cost: 2189884.51\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model Parameters</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>60.531874</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-3.730982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-3.265008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-4.293987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-3.048819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-1.617722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-1.801005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-3.069723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-3.551792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>-2.751712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>-2.019922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>-2.086688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>-1.761378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>-0.340730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.214782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>-1.184080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.614349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>-3.730982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>-3.730982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>-40.780573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>-40.780573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>-40.780573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>-1.502418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>-1.502418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>-1.502418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>-1.502418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>23.068044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>23.068044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>23.068044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>23.068044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>129</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>6.423522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>138</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>140</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>0.052387</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>152 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Model Parameters\n",
       "0           60.531874\n",
       "1           -3.730982\n",
       "2           -3.265008\n",
       "3           -4.293987\n",
       "4           -3.048819\n",
       "5           -1.617722\n",
       "6           -1.801005\n",
       "7           -3.069723\n",
       "8           -3.551792\n",
       "9           -2.751712\n",
       "10          -2.019922\n",
       "11          -2.086688\n",
       "12          -1.761378\n",
       "13          -0.340730\n",
       "14           0.214782\n",
       "15          -1.184080\n",
       "16           0.614349\n",
       "17          -3.730982\n",
       "18          -3.730982\n",
       "19         -40.780573\n",
       "20         -40.780573\n",
       "21         -40.780573\n",
       "22          -1.502418\n",
       "23          -1.502418\n",
       "24          -1.502418\n",
       "25          -1.502418\n",
       "26          23.068044\n",
       "27          23.068044\n",
       "28          23.068044\n",
       "29          23.068044\n",
       "..                ...\n",
       "122          6.423522\n",
       "123          6.423522\n",
       "124          6.423522\n",
       "125          6.423522\n",
       "126          6.423522\n",
       "127          6.423522\n",
       "128          6.423522\n",
       "129          6.423522\n",
       "130          6.423522\n",
       "131          6.423522\n",
       "132          6.423522\n",
       "133          6.423522\n",
       "134          6.423522\n",
       "135          6.423522\n",
       "136          0.052387\n",
       "137          0.052387\n",
       "138          0.052387\n",
       "139          0.052387\n",
       "140          0.052387\n",
       "141          0.052387\n",
       "142          0.052387\n",
       "143          0.052387\n",
       "144          0.052387\n",
       "145          0.052387\n",
       "146          0.052387\n",
       "147          0.052387\n",
       "148          0.052387\n",
       "149          0.052387\n",
       "150          0.052387\n",
       "151          0.052387\n",
       "\n",
       "[152 rows x 1 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set up linear regression parameters.\n",
    "num_iterations = 50000  # Number of gradient descent iterations.\n",
    "regularization_param = 0  # Helps to fight model overfitting.\n",
    "learning_rate = 0.02  # The size of the gradient descent step.\n",
    "polynomial_degree = 15  # The degree of additional polynomial features.\n",
    "sinusoid_degree = 15  # The degree of sinusoid parameter multipliers of additional features.\n",
    "normalize_data = True  # Flag that indicates that data needs to be normalized before training.\n",
    "\n",
    "# Init linear regression instance.\n",
    "linear_regression = LinearRegression(x, y, polynomial_degree, sinusoid_degree, normalize_data)\n",
    "\n",
    "# Train linear regression.\n",
    "(theta, cost_history) = linear_regression.train(\n",
    "    learning_rate,\n",
    "    regularization_param,\n",
    "    num_iterations\n",
    ")\n",
    "\n",
    "# Print training results.\n",
    "print('Initial cost: {:.2f}'.format(cost_history[0]))\n",
    "print('Optimized cost: {:.2f}'.format(cost_history[-1]))\n",
    "\n",
    "# Print model parameters\n",
    "theta_table = pd.DataFrame({'Model Parameters': theta.flatten()})\n",
    "theta_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Gradient Descent Progress\n",
    "\n",
    "The plot below illustrates how the cost function value changes over each iteration. You should see it decreasing. \n",
    "\n",
    "In case if cost function value increases it may mean that gradient descent missed the cost function minimum and with each step it goes further away from it. In this case you might want to reduce the learning rate parameter (the size of the gradient step).\n",
    "\n",
    "From this plot you may also get an understanding of how many iterations you need to get an optimal value of the cost function. In current example you may see that there is no much sense to increase the number of gradient descent iterations over 500 since it will not reduce cost function significantly.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XmYHVW97vHv252RJIQhLQKJdNAgRmRsEBEV54AI6lEJ4oACOYpw9YgDqAc5+BwPyr1e5YhiLiIOCCJOuRhFVBAvyBCEBAKCIQSTMKRJAgkJGTr9u3/U6p1K07urE1K9u7vez5P9dNVaq6rW2tndv11rVa1SRGBmZgbQ1OgKmJnZwOGgYGZmNQ4KZmZW46BgZmY1DgpmZlbjoGBmZjUOCtYQkhZJelNa/rykSxtdJzNzULAeSJou6TZJayQtS8unS1IZx4uIr0TEqc93P5JaJYWkYb2UOU/SRkmr0+tBSd+StPvzPX5ZUpte0kv+yZI2SXpG0ipJd0s6tj/raEOHg4JtQdJZwDeBC4EXArsBHwVeDYyos01zv1Vw+/hpRIwDdgHeSdbOOwdyYOiDv0bEWGAn4HvA1ZJ27l6ot4C5Lbb3/qzxHBSsRtJ44Hzg9Ii4JiJWR+auiDgpItancpdL+o6k2ZLWAK+X9DZJd6Vvqoslnddt3x+Q9Iik5ZK+0C3vPEk/zq0fLukWSU9JmivpqFzejZK+LOnm9E3/95ImpOyb0s+n0rfmV/XW3ojYGBHzgROAduCs3HGOTd+4n0p12T+X9zlJS9PxH5D0xpTenLrCHkp5d0qalPL2lXS9pBVpm/fm9ne5pIsl/SZtd5ukF6e8rjbNTW06oaBNncBlwGjgxZKOkrQk1flx4Ptpv6dJWpDqM0vSHrn6vCXV8WlJ35b0Z0mnpryT03v/vyUtB85L6R+RdL+klZKuk7RXSlcquyx9Nu6RtF/KO0bSfanNSyV9ure2WT+JCL/8IiIApgEdwLCCcpcDT5OdPTQBo4CjgFek9f2BJ4B3pPJTgWeA1wIjga+n47wp5Z8H/Dgt7wksB45J+3pzWm9J+TcCDwH7kP3huxG4IOW1AtFb/fPH6pZ+PnBbWj4IWAa8EmgGPgQsSnV/KbAY2CN3zBen5c8A96QyAg4AdgXGpG0+DAxL+38SmJp7P5cDh6X8K4CrcnUL4CW9tOlk4P+l5WHAJ4DVwPj0/9IBfDXVfzTwhnT8g1PafwM3pe0nAKuAd+X2tRE4NXesDuDMlD8aOB5YALwspX0RuCWVfytwJ9kZjFKZ3VPeY8Br0vLOwMGN/h3wKwbnmYKky9I3j3v7UPZFkm5I32LnSTqmP+o4SE0AnoyIjq6E3Df2ZyW9Nlf21xFxc0R0RsS6iLgxIu5J6/OAK4HXpbLvBq6NiJsiO9v4d6CzTh3eD8yOiNlpX9cDc8iCRJfvR8SDEfEscDVw4HZo+6Nk3UkAM4DvRsRtEbEpIn4ArAcOBzaR/SGdKml4RCyKiIfSdqcCX4yIByIzNyKWA8cCiyLi+xHRERF3AT8H3pM7/i8j4vb03l+xDW06XNJTwOPAicA7I+LplNcJfCki1qf37CTgsoj4W/r/OAd4laRWsvd5fkT8ItXlorTPLd6riPjv1JZnyboX/ysi7k/bfAU4MJ0tbATGAfsCSmUeS/vZmN7HHSNiZUT8bSvbbCUYlEGB7JvVtD6W/SJwdUQcBEwHvl1WpYaA5cCEfD9xRBwRETulvPznZXF+Q0mvTMG3XdLTZH8ourp19siXj4g1aX892Qt4TwpET6U/dEcC+f7+/B+ptcDYrWlkHXsCK3J1OKtbHSaRnR0sAD5JdsaxTNJVua6XSWRnMT216ZXd9ncS2VjG9mrTrRGxU0RMiIjDI+IPubz2iFiXW98DeKRrJSKeIfv/2JPn/l8FsKTbsRZ3W98L+GaubSvIzgr2jIg/Ad8CLiZ7v2ZK2jFt9y9kQeiR1EXVa3ef9Y9BGRQi4iY2/wIDIOnFkn6X+nH/ImnfruJA14dwPNk3QuvZX8m+ER/fh7Ldp9f9CTALmBQR44FLyP4wQNZNMKmroKQdyLpVerIY+FH6A9f1GhMRF2xDnfpEUhPwduAvuTr8Z7c67BARVwJExE8i4kiyP4ZB1jXTtd2L67Tpz932NzYiPrYt9d0G3d+XR8nqDoCkMWT/H0vJ/q8m5vKUX6+zv8XAv3Zr3+iIuAUgIi6KiEPIuhH3IetmIyLuiIjjgRcAvyI767MGG5RBoY6ZwJnpw/dpNp8RnAe8X9ISYDZZX6j1ICKeAv4D+Lakd0saJ6lJ0oFk/eK9GQesiIh1kg4D3pfLuwY4VtKRkkaQ9d/X++z9GHi7pLemgdtRabC0+x+mnrSTdZXs3YeySBom6WVkXV0vJBvrAPg/wEfT2Y8kjVE2kD5O0kslvUHSSGAd8Cybu8IuBb4saUrabn9JuwLXAvsoG2wfnl6HpmP3xRN9bVMfXQl8WNKBqR1fIRtPWQT8BniFpHekM8aPs+UZTU8uAc6R9HLILliQ9J60fGh6H4cDa8jes05JIySdJGl8RGwkG8eo16Vo/WhIBAVJY4EjgJ9Juhv4Lpu7G04ELo+IiWSnqj9K3wytBxHxNeBTwGfJ/hg9QfZ+fg64pZdNTwfOl7QaOJfct77IrvD5ONnZxGPASp7bJdFVdjHZmcrnyf7ILyb7Zln4fxYRa4H/BG5OXRmH1yl6gqRnyAbLZ5F1nRwSEY+m/cwBTiPr9lhJNoh6ctp2JHAB2UDt42Tfcs9JeV9P7f492R+57wGjI2I18Bay7stH03ZdA799cR7wg9Sm9xYVLpK6lv6dbFzjMbKzm+kp70mysY6vkb0vU8nGdNb3sr9fkrXnKkmrgHuBo1P2jmRBdiVZl9VyssudAT4ALErbfJSsS80aTFmX4eCTBsWujYj9Uh/lAxHxnOvMJc0HpqU/NkhaCBweEcv6s75mg1H6ArUEOCkibmh0fax8Q+Ibc0SsAh7OnbJK0gEp+59A13XkLyO7fLK9IRU1GwRS191OqWvp82RjQ7c2uFrWTwZlUJB0Jdmg6EuV3ZhzCtmp5ymS5gLz2TxYehZwWkq/Ejg5BuvpkVn/eBXZVVRPkg3AvyNdemoVMGi7j8zMbPsblGcKZmZWjkE3mdWECROitbW10dUwMxtU7rzzzicjoqWo3KALCq2trcyZM6fR1TAzG1QkPVJcyt1HZmaW46BgZmY1pQUF9XEm03QbfIekd5dVFzMz65syzxQup2AmU2VP7Poq2bQAZmbWYKUFhZ5mMu3BmWTzr3jKCTOzAaBhYwqS9iR7Pu53+lB2hqQ5kua0t3uGCjOzsjRyoPkbwOcie6ZsryJiZkS0RURbS0vhZbZmZraNGnmfQhvZVLuQPaHrGEkdEfGrMg724BOruXbuo3zwiFYmjO3rjMVmZtXSsDOFiJgcEa0R0Ur2EJbTywoIAP944hku+tMCVqzZUNYhzMwGvdLOFNJMpkeRPfN3CfAlYDhARFxS1nHNzGzblRYUIuLErSh7cln1MDOzvqvcHc2eKdzMrL7KBIVsPNvMzHpTmaBgZmbFHBTMzKymckEh8KCCmVk9lQkKHlIwMytWmaBgZmbFKhcUfEmqmVl9lQkKviTVzKxYZYKCmZkVc1AwM7OaygUFjymYmdVXoaDgQQUzsyIVCgpmZlbEQcHMzGoqFxQ8zYWZWX2VCQq+T8HMrFhlgoKZmRVzUDAzs5rKBQXfp2BmVl9pQUHSZZKWSbq3Tv5JkuZJukfSLZIOKKsu4LsUzMz6oswzhcuBab3kPwy8LiJeAXwZmFliXczMrA+GlbXjiLhJUmsv+bfkVm8FJpZVFzMz65uBMqZwCvDbepmSZkiaI2lOe3v7Nh1AvibVzKxQw4OCpNeTBYXP1SsTETMjoi0i2lpaWvqvcmZmFVNa91FfSNofuBQ4OiKWN7IuZmbWwDMFSS8CfgF8ICIe7K/j+pJUM7P6SjtTkHQlcBQwQdIS4EvAcICIuAQ4F9gV+Hbq7++IiLbS6lPWjs3MhpAyrz46sSD/VODUso5vZmZbr+EDzWZmNnBULih46mwzs/oqExR8m4KZWbHKBAUzMyvmoGBmZjWVCwq+T8HMrL7KBAWPKZiZFatMUDAzs2KVCwruPTIzq68yQUGe6MLMrFBlgoKZmRVzUDAzs5rKBYXwNalmZnVVJyh4SMHMrFB1goKZmRVyUDAzs5rKBQWPKJiZ1VeZoOAhBTOzYpUJCmZmVsxBwczMakoLCpIuk7RM0r118iXpIkkLJM2TdHBZdcnzbQpmZvWVeaZwOTCtl/yjgSnpNQP4Tol1QZ4728ysUGlBISJuAlb0UuR44IeRuRXYSdLuZdXHzMyKNXJMYU9gcW59SUp7DkkzJM2RNKe9vb1fKmdmVkWDYqA5ImZGRFtEtLW0tDzfvW2XOpmZDUWNDApLgUm59YkprRQeUTAzK9bIoDAL+GC6Culw4OmIeKyB9TEzq7xhZe1Y0pXAUcAESUuALwHDASLiEmA2cAywAFgLfLisuuT5klQzs/pKCwoRcWJBfgAfL+v43fmKVDOzYoNioNnMzPqHg4KZmdVULih4SMHMrL7KBAX5olQzs0KVCQpmZlbMQcHMzGoqFxR8n4KZWX2VCQq+T8HMrFhlgoKZmRVzUDAzs5rKBYXwoIKZWV2VCQoeUjAzK1aZoGBmZsUqFxTceWRmVl91goL7j8zMClUnKJiZWSEHBTMzq6lcUPAVqWZm9VUmKHjqbDOzYpUJCmZmVqzUoCBpmqQHJC2QdHYP+S+SdIOkuyTNk3RMmfUxM7PelRYUJDUDFwNHA1OBEyVN7Vbsi8DVEXEQMB34dln16RK+U8HMrK4yzxQOAxZExMKI2ABcBRzfrUwAO6bl8cCjZVXGU2ebmRUrMyjsCSzOrS9JaXnnAe+XtASYDZzZ044kzZA0R9Kc9vb2MupqZmY0fqD5RODyiJgIHAP8SNJz6hQRMyOiLSLaWlpa+r2SZmZVUWZQWApMyq1PTGl5pwBXA0TEX4FRwIQS6+TJj8zMelFmULgDmCJpsqQRZAPJs7qV+SfwRgBJLyMLCqX0D3lIwcysWGlBISI6gDOA64D7ya4ymi/pfEnHpWJnAadJmgtcCZwcfgqOmVnDDCtz5xExm2wAOZ92bm75PuDVZdbhOXXqz4OZmQ0yjR5o7jfyNalmZoX6FBQk/agvaWZmNrj19Uzh5fmVdLfyIdu/OmZm1ki9BgVJ50haDewvaVV6rQaWAb/ulxpuZx7GNjOrr9egEBH/FRHjgAsjYsf0GhcRu0bEOf1Ux+3CQwpmZsX62n10raQxAJLeL+nrkvYqsV5mZtYAfQ0K3wHWSjqA7N6Ch4AfllYrMzNriL4GhY50U9nxwLci4mJgXHnVKo+nzjYzq6+vN6+tlnQO8AHgNWnSuuHlVWv785CCmVmxvp4pnACsBz4SEY+TTW53YWm1MjOzhuhTUEiB4ApgvKRjgXUR4TEFM7Mhpq93NL8XuB14D/Be4DZJ7y6zYmXxfQpmZvX1dUzhC8ChEbEMQFIL8AfgmrIqtr35PgUzs2J9HVNo6goIyfKt2NbMzAaJvp4p/E7SdWTPPIBs4Hl2L+XNzGwQ6jUoSHoJsFtEfEbSu4AjU9ZfyQaeBx0PKZiZ1Vd0pvAN4ByAiPgF8AsASa9IeW8vtXbblQcVzMyKFI0L7BYR93RPTGmtpdTIzMwapigo7NRL3ujtWZH+4kdAm5nVVxQU5kg6rXuipFOBO8upUjl8SaqZWbGiMYVPAr+UdBKbg0AbMAJ4Z9HOJU0Dvgk0A5dGxAU9lHkvcB7ZGPDciHhfn2tvZmbbVa9BISKeAI6Q9Hpgv5T8m4j4U9GO0yM7LwbeDCwB7pA0KyLuy5WZQjaQ/eqIWCnpBdvYDjMz2w76dJ9CRNwA3LCV+z4MWBARCwEkXUU29fZ9uTKnARdHxMp0nGXP2ct25hEFM7P6yrwreU9gcW59SUrL2wfYR9LNkm5N3U2l8JCCmVmxvt7RXObxpwBHkU3HfZOkV0TEU/lCkmYAMwBe9KIX9Xcdzcwqo8wzhaXApNz6xJSWtwSYFREbI+Jh4EGyILGFiJgZEW0R0dbS0lJahc3Mqq7MoHAHMEXSZEkjgOnArG5lfkV2loCkCWTdSQtLrJMHFczMelFaUIiIDuAM4DrgfuDqiJgv6XxJx6Vi1wHLJd1HNpD9mYhYXkZ95BsVzMwKlTqmEBGz6TabakScm1sO4FPpZWZmDeZnIpiZWU3lgkJ4UMHMrK7KBAWPKJiZFatMUDAzs2KVCwqeOdvMrL7KBAVfkWpmVqwyQcHMzIo5KJiZWU3lgoLHFMzM6qtMUJAvSjUzK1SZoGBmZsUcFMzMrKZyQcFDCmZm9VUmKPg+BTOzYpUJCmZmVsxBwczMaioXFMI3KpiZ1VW5oGBmZvU5KJiZWU3lgoI7j8zM6qtMUPAlqWZmxUoNCpKmSXpA0gJJZ/dS7l8khaS2MutjZma9Ky0oSGoGLgaOBqYCJ0qa2kO5ccAngNvKqouZmfVNmWcKhwELImJhRGwArgKO76Hcl4GvAutKrEuNr0g1M6uvzKCwJ7A4t74kpdVIOhiYFBG/6W1HkmZImiNpTnt7+zZVxlNnm5kVa9hAs6Qm4OvAWUVlI2JmRLRFRFtLS0v5lTMzq6gyg8JSYFJufWJK6zIO2A+4UdIi4HBglgebzcwap8ygcAcwRdJkSSOA6cCsrsyIeDoiJkREa0S0ArcCx0XEnBLrhO9UMDOrr7SgEBEdwBnAdcD9wNURMV/S+ZKOK+u49fg+BTOzYsPK3HlEzAZmd0s7t07Zo8qsi5mZFavMHc1mZlasckHB9ymYmdVXmaDgMQUzs2KVCQpmZlbMQcHMzGoqFxQ8pGBmVl9lgoLnPjIzK1aZoGBmZsUqFxR8SaqZWX2VCQq+JNXMrFhlgoKZmRVzUDAzs5rKBYXwRalmZnVVJih4SMHMrFhlgoKZmRVzUDAzs5rKBIWuS1I7PaRgZlZXhYJCFhXCd6+ZmdVVmaDQnIJCp4OCmVldlQkKTSkobOpscEXMzAawUoOCpGmSHpC0QNLZPeR/StJ9kuZJ+qOkvcqqS1NqaacHFczM6iotKEhqBi4GjgamAidKmtqt2F1AW0TsD1wDfK2s+jQ3ufvIzKxImWcKhwELImJhRGwArgKOzxeIiBsiYm1avRWYWFZlat1HDgpmZnWVGRT2BBbn1pektHpOAX7bU4akGZLmSJrT3t6+TZVpqg00b9PmZmaVMCAGmiW9H2gDLuwpPyJmRkRbRLS1tLRs0zFq3UeOCmZmdQ0rcd9LgUm59YkpbQuS3gR8AXhdRKwvqzJNtZvXHBTMzOop80zhDmCKpMmSRgDTgVn5ApIOAr4LHBcRy0qsC01NXZekOiiYmdVTWlCIiA7gDOA64H7g6oiYL+l8ScelYhcCY4GfSbpb0qw6u3vemnzzmplZoTK7j4iI2cDsbmnn5pbfVObx85o90GxmVmhADDT3h66b19x9ZGZWX3WCgifEMzMrVJmg0Oy5j8zMClUmKDR5mgszs0KVCQqQ3avgoGBmVl+lgsLw5iY2dLj/yMysnkoFhXGjhrNqXUejq2FmNmBVKijsOHoYq9ZtbHQ1zMwGrEoFhXGjhrPqWQcFM7N6KhUUWsaO5IlV6xpdDTOzAatSQWHyhB14ZPlaT59tZlZHpYJC64QxrO/o5DGfLZiZ9ahSQeFlu+8IwNzFTzW4JmZmA1OlgsJ+e4xn9PBmbl24vNFVMTMbkCoVFEYMa+J1+7Twf+c+yrqNmxpdHTOzAadSQQHglNdMZuXajZz983ms3eAb2czM8kp9yM5AdGjrLpz15n34X9c/yB//voxXTt6Vl++xI3u3jGHvCWOZtMtoxo8ejtKsqmZmVVK5oABw5huncMRLduXqO5Zw+6IV/PHvT5CfJ2/U8CZ2Hz+aF+44it3Hj2K38aPYdcwIdhkzgp3HjGCXHTYvjxnR7ABiZkNGJYMCwCF77cIhe+0CwLqNm/jnirUsbF/D0qee5fGnn+Wxp9fx2NPruO3hFTy+al3dJ7YNbxZjRg5jzIhhjB05jDEjmxkzMlveYcQwxo5sZvSIYYwY1sTI3Ctbb66ld60PbxbNTZtfw5pEc1MTzRLNzcp+NvVUJsvrmiLczGxbVDYo5I0a3sw+u41jn93G9Zjf2RmsXtfBirUbWLl2AyvXbGDFmrS8diNr1nfwzPoO1qzvYM36TTyzvoMnVq2rLT+7cVO/z84qgcieOJctZwlKeU1SWs5+duU1NW2ZrpTZJGr7UW0/2x6Aetu01zx6P2bv2xbVqX6JXrct2LHDdN/4jLvY9EMncepr9i71GKUGBUnTgG8CzcClEXFBt/yRwA+BQ4DlwAkRsajMOm2LpiYxfofhjN9hOJMZs037iAg2bOpkQ0cn69MrW95US9uQXps6g00R2c/06ugMOtPPTRFs2tTJpoBNnZ1s6sx+dnQGnQFEkH4QZGldy+kfEVumd3Wf1dJTWtd+IOjs3DK9flsL3ovett62rHTc+iWKt9224xY93tX3zveR36g+mTB2ZOnHKC0oSGoGLgbeDCwB7pA0KyLuyxU7BVgZES+RNB34KnBCWXVqJEmMHNbMyGHN9Hw+YmbWeGVeknoYsCAiFkbEBuAq4PhuZY4HfpCWrwHeKJ9Dmpk1TJlBYU9gcW59SUrrsUxEdABPA7t235GkGZLmSJrT3t5eUnXNzGxQ3LwWETMjoi0i2lpaWhpdHTOzIavMoLAUmJRbn5jSeiwjaRgwnmzA2czMGqDMoHAHMEXSZEkjgOnArG5lZgEfSsvvBv4URZdzmJlZaUq7+igiOiSdAVxHdknqZRExX9L5wJyImAV8D/iRpAXACrLAYWZmDVLqfQoRMRuY3S3t3NzyOuA9ZdbBzMz6blAMNJuZWf/QYOvCl9QOPLKNm08AntyO1RkM3OZqcJur4fm0ea+IKLx8c9AFhedD0pyIaGt0PfqT21wNbnM19Eeb3X1kZmY1DgpmZlZTtaAws9EVaAC3uRrc5moovc2VGlMwM7PeVe1MwczMeuGgYGZmNZUJCpKmSXpA0gJJZze6PltL0mWSlkm6N5e2i6TrJf0j/dw5pUvSRamt8yQdnNvmQ6n8PyR9KJd+iKR70jYXNfq5FpImSbpB0n2S5kv6REofym0eJel2SXNTm/8jpU+WdFuq50/TXGJIGpnWF6T81ty+zknpD0h6ay59QP4eSGqWdJeka9P6kG6zpEXps3e3pDkpbWB8tiNiyL/I5l56CNgbGAHMBaY2ul5b2YbXAgcD9+bSvgacnZbPBr6alo8Bfkv2eODDgdtS+i7AwvRz57S8c8q7PZVV2vboBrd3d+DgtDwOeBCYOsTbLGBsWh4O3JbqdzUwPaVfAnwsLZ8OXJKWpwM/TctT02d8JDA5ffabB/LvAfAp4CfAtWl9SLcZWARM6JY2ID7bVTlT6MtT4Aa0iLiJbNLAvPyT634AvCOX/sPI3ArsJGl34K3A9RGxIiJWAtcD01LejhFxa2SfqB/m9tUQEfFYRPwtLa8G7id7KNNQbnNExDNpdXh6BfAGsicTwnPb3NOTC48HroqI9RHxMLCA7HdgQP4eSJoIvA24NK2LId7mOgbEZ7sqQaEvT4EbjHaLiMfS8uPAbmm5Xnt7S1/SQ/qAkLoIDiL75jyk25y6Ue4GlpH9kj8EPBXZkwlhy3rWe3Lh1r4XjfYN4LNAZ1rflaHf5gB+L+lOSTNS2oD4bJc6S6r1n4gISUPu+mJJY4GfA5+MiFX5rtGh2OaI2AQcKGkn4JfAvg2uUqkkHQssi4g7JR3V6Pr0oyMjYqmkFwDXS/p7PrORn+2qnCn05Slwg9ET6VSR9HNZSq/X3t7SJ/aQ3lCShpMFhCsi4hcpeUi3uUtEPAXcALyKrLug6wtcvp71nly4te9FI70aOE7SIrKunTcA32Rot5mIWJp+LiML/ocxUD7bjR5w6Y8X2RnRQrIBqK7Bppc3ul7b0I5WthxovpAtB6a+lpbfxpYDU7fH5oGph8kGpXZOy7tEzwNTxzS4rSLrC/1Gt/Sh3OYWYKe0PBr4C3As8DO2HHQ9PS1/nC0HXa9Oyy9ny0HXhWQDrgP69wA4is0DzUO2zcAYYFxu+RZg2kD5bDf8g9CP/xHHkF3B8hDwhUbXZxvqfyXwGLCRrI/wFLK+1D8C/wD+kPtACLg4tfUeoC23n4+QDcItAD6cS28D7k3bfIt0t3sD23skWb/rPODu9DpmiLd5f+Cu1OZ7gXNT+t7pl3xB+mM5MqWPSusLUv7euX19IbXrAXJXngzk3wO2DApDts2pbXPTa35XnQbKZ9vTXJiZWU1VxhTMzKwPHBTMzKzGQcHMzGocFMzMrMZBwczMahwUrHIkPZN+tkp633be9+e7rd+yPfdvVjYHBauyVmCrgkLuLtt6tggKEXHEVtbJrKEcFKzKLgBek+a0/7c0Gd2Fku5I89b/K4CkoyT9RdIs4L6U9qs0mdn8rgnNJF0AjE77uyKldZ2VKO373jTP/Qm5fd8o6RpJf5d0Rdfc95IuUPY8iXmS/me/vztWSZ4Qz6rsbODTEXEsQPrj/nREHCppJHCzpN+nsgcD+0U2LTPARyJihaTRwB2Sfh4RZ0s6IyIO7OFY7wIOBA4AJqRtbkp5B5FN0/AocDPwakn3A+8E9o2ISBPkmZXOZwpmm70F+GCauvo2smkHpqS823MBAeB/SJoL3Eo2KdkUenckcGVEbIqIJ4A/A4fm9r0kIjrJpvNoJZsSeh3wPUnvAtY+79aZ9YGDgtlmAs6MiAPTa3JEdJ0prKkVyqZ4fhPwqog4gGy+olHP47jrc8ubgGGRPSvgMLIHyRwL/O557N+szxwUrMpWkz3qs8t1wMfSlN1I2kfSmB62Gw+sjIi1kvYlm42yy8au7bv5C3BCGrdoIXu86u31KpaeIzE+ImYD/0bW7WRWOo8pWJXNAzalbqDLyebxbwX+lgZ72+n5MYa/Az6a+v0fIOtC6jITmCfpbxFxUi79l2TPRphLNvvrZyPi8RRUejIO+LWkUWRnMJ/atiaabR3Pkmqi6uhwAAAAMElEQVRmZjXuPjIzsxoHBTMzq3FQMDOzGgcFMzOrcVAwM7MaBwUzM6txUDAzs5r/D50bojQfNKZqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot gradient descent progress.\n",
    "plt.plot(range(num_iterations), cost_history)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.title('Gradient Descent Progress')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Model Predictions\n",
    "\n",
    "Since our model is trained now we may plot its predictions over the training and test datasets to see how well it fits the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xd8k9X+wPHPaRugBaQFKiNMGUUQoVAVBReOIkMqW0FRcP1Er6LWW1wol3sFcV/XRRwIqEzLlDpAEQRltCwBLVMCSBEKAgU6zu+PJylpkzRpmzSj3/frxYvy5GlyHtJ+c55zvud7lNYaIYQQoSvM3w0QQgjhWxLohRAixEmgF0KIECeBXgghQpwEeiGECHES6IUQIsS5DfRKqY+UUoeVUlucPPaEUkorpepa/62UUm8ppTKVUpuUUp180WghhBCe86RH/wnQo/hBpVRj4GZgn93hW4BW1j/3A++Vv4lCCCHKw22g11qvAI46eeh14CnAfsVVX+BTbVgDRCulGnilpUIIIcokoizfpJTqC1i01huVUvYPmYE/7P6933rsYEnPV7duXd2sWbOyNEUIISqt9evXH9Fax7o7r9SBXikVBTyNMWxTZkqp+zGGd2jSpAnr1q0rz9MJIUSlo5Ta68l5Zcm6aQE0BzYqpfYAjYANSqn6gAVobHduI+sxB1rryVrrBK11Qmys2w8kIYQQZVTqQK+13qy1vlBr3Uxr3QxjeKaT1voQsAC4y5p90wU4rrUucdhGCCGEb3mSXvk5sBqIU0rtV0qNLOH0JcAuIBP4AHjIK60UQghRZm7H6LXWt7t5vJnd1xoYVf5mCSGE8BZZGSuEECGuTOmVQgSa1HQLk9J2YMnOIVwp8rXGHB1JcmIcSfFmfzdPCL+SQC+C3rOpm5mxZl/hyr18665pluwcxszbDCDBXlRqMnQjglpquqVIkC8uJzefSWk7KrRNQgQa6dGLoGI/RKPAZYC3dyA7x9fNEiKgSaAXQSM13cKYeZvJyc0HPAvyAA2jI33XKCGCgAR6ERRS0y08MWtj4fi7pyJN4SQnxvmoVUIEBwn0IuDZevKlDfKSdSOEQQK9CHiT0nYUDtd4IjrSRMbYctXcEyKkSNaNCHilmUw1hSteuLWdD1sjRPCRHr0IeA2jI7F4EOxjokyM7WME+a4TlnEgO4eGMnwjhAR6EZhsaZQHsnOoFWlyeZ45OpJVKd2LfJ99Zo4smhJCAr0IQMWDdXZOLmFAQbHznGXU2MbzG544zP0/z+PqPRlE5p7h91kt4M0XoIfD9sdChDwZoxcBx9nkawEQaQrDHB2JwujJv9SvvUMv/UB2DrdsX8k3Ux7i9o1p7KrdkNVN2tPq4E645Ra4+244d67CrkWIQCA9ehFwXE2+5uQWuB1vH7HrR56bP5H1DdvwSN+nOHDBhQBE5Ofx9Ia5jJg6FY4fh1mzwOR6SEiIUCI9ehFwSlrJWmLdmmXLeGbeK6xs2oHbb3+pMMgD5IVHMO6ywfDWW5CaCmPGeLPJQgQ0CfQi4JS0ktWSnUNqupNtiI8cgWHDCGvdmpQ7xnIuwrG3roDUbv3goYfg1Vdh0SIvtlqIwCWBXgScpHgzMVGuh1XGzNvsGOwfeQT++gs++4wnB16OcvJ9GusdwWuvQfv2RsA/edKrbRciEEmgFwFpbJ92RJrCnT7mUHr4++/hiy+M4ZiOHUmKN7sseHYgOweqVoX334c//oBx47zediECjQR6EXBsOfQllT0onLDNy4N//AOaNoV//rPwcbOLcX6NsZgqNbIpDB9ujNlbnAwFCRFCJNCLgGLLoXe3ErZwwvbzz2HzZnjlFYg8H9yTE+Nc3hHYFlF9PfBBKCiA8eO91n4hApHbQK+U+kgpdVgptcXu2CSl1Hal1Cal1JdKqWi7x8YopTKVUjuUUom+argITZ4UMCtcKJWfbwTpDh2gf/8i5yTFm3mpX3uXPfuc3Hxe3Hwa7r0XpkyBPXu8dQlCBBxPevSfAMWXE34DXKK1vhT4DRgDoJRqCwwB2lm/512llPNulRBOlNSTd1goNXMm/PYbPPccKMfp16R4M6tSujudmAXr8M/TTxv/eOut8jdeiADlNtBrrVcAR4sd+1prnWf95xqgkfXrvsAXWuuzWuvdQCZwuRfbK0JYarrFZVA2R0eye0IvVqV0N4K8bcjlkkvgtttKfF5XefkNoyOhUSMYPNjo1R8/Xs4rECIweWOMfgTwlfVrM/CH3WP7rceEcOvFhVudZssonOTWf/01bNtmTMCGlfxjfH2bWKcfIKfP5RlpmqNHw99/G8FeiBBUrkCvlHoGyANmlOF771dKrVNKrcvKyipPM0QISE23cOx0rtPHNE4qT/73v1CvHgwa5PZ55663OP0AOXY618jJD6sPV18N775r3CkIEWLKHOiVUncDvYGhWhfu8WYBGtud1sh6zIHWerLWOkFrnRAbG1vWZogQUVJpA4cJ1cxM+OorePBBqFLF7fOWNLlbmJP/wAOwaxcsX16qdgsRDMoU6JVSPYCngFu11qftHloADFFKVVVKNQdaAb+Uv5ki1JW0i5TDsM0770B4uBGcy/G8Rc7p3x9iYuCDD9yeL0Sw8SS98nNgNRCnlNqvlBoJvA3UBL5RSmUopd4H0FpvBWYBvwJLgVFaa883+xSVlqsJ0+hIU9Fhm1On4OOPYeBAaNCgzM/rcE61anDnnfDll0bdHCFCiCdZN7drrRtorU1a60Za6w+11i211o211h2tfx60O//fWusWWus4rfVXJT23EGCMo586m+dwPNIU7rj/a2qqkR3z4IMO5ztT0sIp22sU3jHce69Rq37aNI/bLkQwkJWxwq9sK2Gzc4pOxMZEmZxuLML06Ua5g27dPHp+28KpaCfbESqgf2fz+ddo3x4SEozXECKESKAXfvXiwq1OJ0ujqkQ4BvlDh4y0ymHD3KZU2kuKN1O9quMeOxpYvr1YxtfQobBhA2zf7vHzCxHoJNALvykppdLpJOoXXxjpj0OHlvq1XE3KOhwfPNj4EPnss1K/hhCBSgK98JuSUiqdTqJOmwadO8PFF5f6tUpcHWuvQQPo3t0I9NpVsWMhgosEeuE3pUqp/PVXY0jlzjvL9FrOJmWLTMTau+MO2LkTfpHMYBEaJNALv/E4pRJgxgwjd37IkDK9ln01S4fiaMX162dsTiLDNyJEKB0At6cJCQl63bp1/m6GqGC2jBv7ydhIU7hjAC4ogObNoV07WLLEK687KW0HluwcwpUiX2vM0ZEkJ8adf90BA+DHH+HAAeMDRogApJRar7VOcHee9OiF33jcy165EvbtM7Jtyqn4xib51o6ObTOSwr1ohwyBw4eN1xYiyDnmnAlRgZLizc6HT+xNmwbVq0PfvuV+vZJq39jq3iTFm+GWW4wdq+bMgWuvLffrCuFP0qMXge3MGZg926hFU716uZ/OXe2bwserVzeC/dy5UtFSBD0J9MJvUtMtdJ2wjOYpi40Nu9OdFDpdvNgoeeCFYRtwX/umyOMDBsDBg7B6tVdeWwh/kUAv/MJ+rFzjZIzcZtq087ntXlBS7RuHdMtevYwyyHPneuW1hfAXCfTCL5yNlRfWhrf56y8jy+aOO7yW+VJ80/Bw616zTieCL7gAEhONcfoAyE4ToqxkMlb4hatNwIuMoc+eDbm5Xhu2sfFoAthmwABYuBDWroXLZftjEZwk0IsKZ9sE3FkfucgY+bRpxubfHTr4tC2T0nZwIDuHhsVz6QH69IGICKNXL4FeBKngHro5cQJWrICzZ/3dElEKk9J2uN8EfNcu+OknozevnG3tXX4ezRPExMCNNxrj9DJ8I4JUcAf6r74ycpx3uC6OJQKPqxTHIpuAT59uBPg77vBZOzyaJwBj+GbXLsjI8FlbhPCl4A70bdsaf//6q3/bIUrFVYpj4SbgWhuB/rrroHFjp+d6g6sPHIf5g759jcngOXN81hYhfCm4A33r1kbtcAn0QcVtJcm1a+H3370+CVucqw8cBUWHb+rWNT50Zs+W4RsRlII70FetCi1bSqAPMm5r3EybZmzW3b+/T9uRnBiHs9F/jZNa+QMGGB8+W7b4tE1C+ELwZ920bSuBPgi5THHMzTV2krr1VqhVy+dteGym83F3h2Gd226DUaOM4Zv27X3aLiG8zW2PXin1kVLqsFJqi92x2kqpb5RSv1v/jrEeV0qpt5RSmUqpTUqpTr5sPGAE+t9/h3PnfP5SogKkpcGRI2XeYKS0zJ7uPFWvHlxzjTF8I0SQ8WTo5hOgR7FjKcB3WutWwHfWfwPcArSy/rkfeM87zSxB27aQlweZmT5/KVEBpk+HOnWMFakVoFQ7Tw0YANu2yR2kCDpuA73WegVwtNjhvsBU69dTgSS7459qwxogWinVwFuNdcqWebN1q09fRlSAEydg/nyjFrzJVCEvWeqdp5SS7BsRdMo6Rl9Pa33Q+vUhoJ71azPwh915+63HDuIrF19srFxMT4eBA332MqICzJljlCX2cbZNcR6XRGjQALp1M4Zvnn/e9w0TwkvKnXWjjb0IS51zppS6Xym1Tim1Lisrq9Svaytx2+yF79hSpyk/fvaV61K3IjhMnWqkzF5xhb9b4tqAAUbmzfbt/m6JEB4ra6D/0zYkY/37sPW4BbBf4dLIesyB1nqy1jpBa50QGxtbqhcvvh3cpvqt6HDodw4cO+W81K0IfLt3G+Ushg/3WckDr+jXz/hbSheLIFLWQL8AGG79ejgw3+74Xdbsmy7AcbshHq8pvnR9Y4NWXHD2FM2OHSQnN58XFsh4fdCZNs0I8BU8bFNqjRrBVVdV6Di9Rxu0CFECt2P0SqnPgeuAukqp/cBYYAIwSyk1EtgLDLKevgToCWQCp4F7fNBmhxznjQ1aA3Dpwd/YXdtMdk4uqekWz0vRigrhslKk1vDpp3D99dCkib+b6d6AAfD440amV8uWPnuZ1HQLLyzYSnZObuExW+E1QH6+hcc8ybq5XWvdQGtt0lo30lp/qLX+S2t9g9a6ldb6Rq31Ueu5Wms9SmvdQmvdXmu9zheNLp7j/HvdJuREVKXjwd8Kjz0xa6P0gAJIiZUiV62CnTuNYZtgYFux68Neve3/yz7I2zgtvCZECYJyZWxyYhxj5m0uHL7JDwtngzmOLvs2F56Tb61JIj2gwFBSpcikXV8Ym3Hbxr/9wHa3YcnOKVIrPybKxNg+7Yr+7DRpYkwYz54NKSnOnq7cnP1/2XO3ybkQ9oKy1k3x7eAAVjaL5+KsPcSePOZwvvSA/M/VjlJHsrJh5kyjl1yjRgW3ylB8ct8+hezY6VyS52x0vCscPBg2bPBZ9o2r/y8bd5ucC2EvKAM9GMF+VUp33hjckUhTOD82iwfgqr0e1i4RFca2o5QzPX5fbSyU8uOwjbvec26+duwoDBliVE6dMaPcr1+YKpyymBZjltAsZXGJ57tcuSuEC0Eb6G1svfsDTVtzrFpNrt7jPNBLD8h/XO0oBXB7xlL2125olAH2E086AZbsnKLzPQ0aGDtPTZ8OBQVlfu3idxP5bsogx0SZXK/cFcKFoA/0NjkFilXNOnLtrvWEFRTtnSmc/KKKCuMqkLb46w+6/LGF6e1vNnrHfuJpJ8Bhq8E774Q9e4wtD8vI3d1EcQ7zBUJ4ICQCve2XZUlcV2JPZ3PFH+drhttPrDndE1T4nKtAenvGUs6FRfBjt94V3KKinBU2c6XIfE9SEkRFGWsAyqi0Q4ry8yvKIiQCve2XZVmLBE6ZqtFn24rCx4rfCMvEbMW7vk2swxh91bxz9N+yjO/iruS+/l380i6b4pP77tblFgbnGjWMOvWzZpV5g/roqNIVb8vJzZfUYVFqIRHobT3GM6ZqLG19JX22raD62dMuz3eX0SC8JzXdwtz1FocP3B47VhFz5m9q/mNUQAxF2Cb390zoxe4JvdgzoZdntervvBOys2HhwlK/Zmq6hZNn8kr9fflaO65FEKIEIRHo7TMQPu3Um5rncui/5TuX5zvsCSp8xtUY9IjNadCiBd3uD8yKo6npFk6ddQzCDhkvN95obGA+eXKpX2NS2g5yC8q3B63coQpPhESgT4o3E2O9Bd7YMI4NDeN44Od5VM1zvuuU0z1BhU84G4Nuf/B3OuzdAg8/7NdJWFdcrUp1mvESHg733gvffGOs7i0Fb6X8yh2qcCfwfsvKaGyfdoUTapOuuQvz31mMWDff5fmSV18xnE3EjlyXyqmqUTBihB9a5J6ru5CoKhHOh5lGjjQCfil79aVN+XVV1FPuUIU7IRPo7SfUVjftQFqrLjy2cgZxWXucnh+mVJEJLakQ6BvFM1rq/X2EXttXcrD/HXDBBX5smWuuOgEue85mM/TpAx9/7PHexa6GhpyJNIUzrEsTIlxEerlDFe6ETKCHohNqE5NGc7xaTT6a/SKNjv/pcK79hFby7I0kz9novOCWKJfiW/WN+jWNCDQtxz/t76a5VFJPu5mrjsADD0BWlkd16ksqWFacbWvD5duzShzPlztUURKl3azEqwgJCQl63TrvFrpMTbcw9d1UPpn2T/LCwvnXDfcxv+21aOX5Z5s5OpJVKd292q7KwmlJ4rgYY+LyuusCeuOO1HQLo2dmlLhtWqQpvOh4fUEBtGkDtWrBL784HWdxVnbY0+dvnrK4xPZER5rIGHuz2+cVoUUptV5rneDuvJDq0dtLijcz/KEkHhj1DpZaF/LGolf56d17mLjkTe5et4Brd62nybGDhBdIhUBvc1aS+LGZGUzq/zgcPWrUcg9gSfFmt3tjOmS7hIXBE0/AunXwww8O56emW0ievdFtkHe1Qbm78fzsnFzix30td6HCqZDt0RdRUMD/9X+WpK3LSdj/K3VyThQ+lBsWzh+16rGzTmO+a3EZS9p040Q1o4qi9OjLpuuEZQ7j2VXzzvHD/+5lb51GHPxySUDkzpfE2TUUp4DdE3qdP5CTA82aQefOsGRJqZ+vpJ8324enu3IJDncaIqR52qMPynr0pRYWxqYrbuSruK6gNbGnsml2zEKzYwdpduwATY8dpP2fmdyU+TPPLv+QafG9mHLN7SQndvR3y4OSszuhgZu/pf7Jozze63H2pu0I+EBUfM8DZxx62ZGR8I9/wLPPwsaN0KFD4UPu7g7dVaS0/X/Zaua7UljjP8D/f0XFqhyBHuMXd/TMDLRSZNWIIatGDGsbX3L+BK3pcHgnI37+kgd+mcvwvauJ6vEpIL8wpdUwOrJIMDLl5/Lgmtmsb9iGn5p2QAXBkFjxwGpfMwlKCMwPPQSvvALPPAOLFhUeLv5/Yi9cKY964UnxZpLizW7vDmTIURQXsmP0xSXFmxnapYnLOiZKKTbWa8nLdz3Pjx+nEhVdE26+Gd57r0LbGQqKp1QO3PwtjU5k8d+rhoBSQVMy2j6L6/XBHYmOPF+XpprJxa9OTIyx69TixUXG6pMT4zCFOf70mcIVrw7qUKoeuLsibMHy/ysqTqUJ9ADjk9rz+uCOhTVMwq2ZEcUrXD6YWYVxz3/MylaXwUMP8b/EkTLJVQq2lMroSBOR587w2MrPWGtuy/cXdQ7qTTPO5p2vO3/sdK7rFNx//MPIrf/nP0ndsJ+uE5YxemYG1atGEGX3ARETZWLSgNIFeSj6/+vM6XN58vMqiqgck7ElKOk2OKwgn1eWvEG/rcuZcON9tHl5rIx9ltKvo/5J23dfpv/QSRy6pJORZhmE/4eufk5cTaCmj32F+HHJPNnzMea0v7HwuLcnS12lbMqkbOVQIemVSqnRSqmtSqktSqnPlVLVlFLNlVI/K6UylVIzlVJVyvMavlbSWGdBWDjJPR9jUVw3nvp2Cqtf/7gCWxa8bKuMO/3jM5p++DYHrk9k7vQnWZXSPWgDT2lWy6amWxia35a15rY8s+xD6pzKLnzMVmbYWz3upHgz1as6TrVJsTNhr8yBXillBv4BJGitLwHCgSHAROB1rXVL4Bgw0hsN9YWS9jK1yQ8L54leo9lcvyXPzfwP3812XRVTwLOpmxk9MwNLdg6jV86g2rkz3BfXL+iHElyNezurMzMpbQen8zQpPR4hKjeHf3/9DtjdOedrzeiZGa5X2ZaSqw8hmZQVNuUdo48AIpVSEUAUcBDoDsyxPj4VSCrna/hMSXuZ2jtrqsr9/Z7htKkaTR+6h4U/Zfq8bcEoNd3CjDX70MAlhzIZmv4Vn3bqxdZa5qDvXSYnxjntFGjgsZkZhZt62w/x7KzbmJevGU6P31YzsliBPW/ueubqQ0gmZYVNmQO91toCvALswwjwx4H1QLbW2lataT8u8hOVUvcrpdYppdZlZWWVtRnlUpoez5816zK69xO0PPIH5554woetCl62D06lC/jX1+/xV/VavN5tKBD8vUt3q2Vtm3rbUjFtPrwsiaWtryTl+4+5budap99b3mEWZzt4BfOkt/C+8gzdxAB9geZAQ6A60MPT79daT9ZaJ2itE2JjY8vajHIpbY9nVbOOTL7sNvqvWVCmHYVCnS2YD9z0LfEHd/DSdfcUrjIOhd6lqx2niivygaAUT/YczfbYZryf+hJX7clw+j1l/SB0toOXAvp3NgftfIjwvvIM3dwI7NZaZ2mtc4F5QFcg2jqUA9AICNjB2dJsCm3zyjV38euFzTl8+3CuHjMv6MeevSk6ykTdU8dI+eET1prbMq+dkY2iICR6l66Gb9w5WTWKuwaNY290fT6Z/QJDMpYWGbOHsn8QOqudr4Hl2/1zlywCU3kC/T6gi1IqSimlgBuAX4HlwADrOcMB17t/+FnxErrm6EjeGNyxxP1Cz0WYeOqWR6lz+jgPLv2A5Dney6AIZqnpFk7m5DL+63epfi6HlB6PFFZwHNqlSUj0Lj0pdubKsahaDBr6MqubXsqEtLeZMnccTY8dAMo3zCITscIT5Rmj/xlj0nUDsNn6XJOBfwKPK6UygTrAh15op8/YVj/untCrSPpfSb39LfVb8lHCrQzNWEqHvVuCfqLRGyal7eCWrT/Q47fVvHb1UHbWbQwY5XPHJ7X3c+u8x9PhG2deHN6Na3/7mS2jn+OqP7awfPIDTF/wb6abtpN0IQ69fE/IRKzwRLmybrTWY7XWbbTWl2it79Ran9Va79JaX661bqm1Hqi1PuutxlYkW28/3MWuPq93G8r+Cy7kpaVvk/XXCafnVCbn9h/gxW/eZ0PDOD647LbC48c9qL0eTMoy3AfGB0RSvBkiIrjktXFE7dlJ2LPP0O3YbjqPHQ2NGkHdutCtm7HF4ksvwZw5sGkTnD5d6vbI6lhhr1KVQCitpHgzrw7q4PwXqUokz938f7T66w8e3LjYD60LIAUFvJv2OtXyzpHc8zEKws7/f4Vaz9J+uM9TTodmGjSAf/0LLBbYsAHefBMGDjTq2n/1FTz9tPHvDh2MzUxuuAHefRdOnnTanuLlEEos0SAqHQn0bpRUV2R5i8v4rsVl3PfDDG5M/qLy/lJNnMhlmRv4T+KD7KzTuPBwqKb42Yb7PAn2zjYRKSIsDOLjjfo4778PK1bAwYNw4oTxAfDFF8ZGLYcOwahRxg5dkyZB7vk7JVkdK9yRQO+BpHgzGWNv5o3BHR12iPtX93upmpfL/WkfVs4e1KpV8NxzMHgwncc9WWRiO9RrrbgaNomJMhVO6pe57EPNmsYHwODBMHEibN0KP/0EV10FTz0Fl10GO3cWnl7qDc1FpVLpi5qVlrO9O1OWf8SDv8zj1rteY1OD1sREmRjbp11IBznA2BYwPh4iIozeZ61a/m5RhXO6N66v3/fUVBg50tindt48uP56l0XXFPD64I6h/7NYSXla1EwCfSk5+4WqcfY0yz+4n3216tN/2CRQClO4KlMJ2qCRnw+9esGyZUav/rLL/N2iymX3bujTB3btgoULSa3dxuWG5rIlZuiq9JuD+4qzMeeTVaN4+ZrhdD6wnaRfvwcgN1+H9vjos89CWhrpT/2Lrt/9TXMvFegSHmreHJYvhxYt4NZbSeKwyxx/yakXEuhLKSneTEyU48TsnPY3kNGgFWO+/5ioc8YvVsj+gs2aBRMmsLvfUO5QHbBk56AxxoNHz8zg2dTN/m5h5RAbC99+C3XqQN++tA93/vMWaplPovQk0JfB2D7tHCbhtArjxRseoN7Jo4xaPQuAMKUKe7rPpm6m64Rlwd/z3bQJ7rkHrrySe+KHOV1+P2PNvuC9vmBTrx7Mnw9HjvDht28SFeG47kNy6oUE+jJwlUudbm7D3HbXc+/aL2ly7CD5Whf2dKev2Vek5xuUGTpHj0JSkjHpOncue0/mOz1NQ2gPWwWa+Hh4800u/GUlM8+slZx64UACfRnZbxz9ht3G0ROvvZu8sAieWV5y5Yegy3HOz4fbbzcW+MybBw0alDgkELLDVoHq3nvh1ltp//ZLxP19yOHhoPt5E14lgd4L7PPs/659Ie9cOYjE39fQ1UVJWpugCoZPPglffw3vvANdupCabuHU2TyXp8u4cAVTCv73P6halYfmvem0bk5Q/bwJr5JA70UvLtxKTm4+H16WxN7o+oz9djIR+SEQDCdPhjfeMFZv3nsvqekWxszb7LAhtU2orogNePXrw7hxXLt7Az1++8nh4aD5eRNeJ4HeS1LTLRw7bQS+sxFVGN/9Xlr/tY9h6Uucnh80wXDZMmPpfY8e8OqrgPMa6DaVYUVsQBs1iuOt2/L8silUzTtXeFhh7EQlKicJ9F5SfPzzm5ZXsKJZPKNXzqD26eNFHouJMhUJhqnplsDMyPntN+jfH+LiWDTmNbq+soLmKYtdLqtXUPYl/8I7IiKo9d5/aXgiq0gnQwNz11sC52dLVCgJ9F7iMP6pFONuuI/q53J44sdpRR6KqhJRJMiPmbc58DJyjh6F3r0hIoKvJ3xA8jd7C9voSi0nhd+EH3Tvzi8tOzNq9SxqnD1f4lgmZCsvCfRe4mz8M7NuEz7t1JvbM9Jodyiz8Lh9j9jZMIjffyFzc40SuXv3QmoqL27JcTlUY++U5GsHjPFXDaN2zgnu++XLIsdlQrZykkDvJc4qGSrgjW53cKR6NBOW/pfwgvPBslnKYpqVMAzit19IreHhh42x+SmUjN6fAAAc20lEQVRToGtXj9sS8mUfgshfF1/KV62v4p71C4r06uWuq3KSQO8lzvafHdqlCX9Xq8HzNz1I+z93MnJtqsfP1zA60j9j92++aWTZjBkDd95Z2BZPSY8xMCQnxvG/qwZxwdlTRcbq5a6rcpJA70XF958dn9QeDSyN68rS1lfy+MoZhRtClyTSFM71bWIrfuz+yy+NTS769YPx4wsPl2b7PEnhCwxJ8Wb2Nr+YFc3iGbkulaq5xo6ectdVOUmg9zFbmYTnb3yQc+EmXlr6NkoXuDxfAdVMYUxfs69ix+5/+gnuuAOuuAKmTTN2PrLydPu8oEkZrSSyT+fyXpeBxJ7KZuCW7wqPy11X5SOB3sdsveHDNevw7+tHcNW+TYxYO9/l+RoK8/Gd8ckv6Y4dRm3zxo1h4UKIinI4JSneTHJinMvN0sOVkvz5ANMwOpLVTdqT3iCO+36ZR5h1jkjuuiofx40mS0EpFQ1MAS7BiFEjgB3ATKAZsAcYpLU+Vq5WBjFb4JuUtoOZl95M951r+ecPU1nT9FK21mtR6ufz+i/pn3/CLbdAeLixKXXdukDRnZNqRZo4l5fP6VzndyKRpnAJ8gEoOTGOMfM288Hlt/Hu/Alcv2sdP118ldx1VULl2mFKKTUV+FFrPUUpVQWIAp4GjmqtJyilUoAYrfU/S3qeYNphqrwWL99CQt/rOGmKpPfwN8ipUs3j7/V6QD1+HG64AbZtg++/L9wlypbb70lKZbhSvDoohHfSCnKp6RZeW7KVLyYOxVKvCZZZC+S9CiE+30pQKVULyAAu0nZPopTaAVyntT6olGoAfK+1LrELUZkCPcAj973Km1OSWXjxNTza50kcdhx3wmzdjxTghQVbC+vMlHl/2pMnITER1q419iDt2bPwIVf7jzqjgN0TepXutUXF+89/4JlnjE3G27b1d2uEl1TEVoLNgSzgY6VUulJqilKqOlBPa33Qes4hoJ6LBt6vlFqnlFqXlZVVjmYEn0V12vDKNXfSd9sPPPjz3BLPjTSF88bgjiQnxvHCgq08NjOjSDGxY6dzeWxmBvHjvvY8IycnB/r2hTVr4PPPiwR5KN08gIz3Br7UdAu9TrfhbLiJeSPGSHplJVSeQB8BdALe01rHA6eAFPsTrD19p7cMWuvJWusErXVCbGzlKrbUMDqSd7sMZGGbq3nqh6nckPmz0/NsE5xAidUiwQj4Hm3jl5NjpE8uXw5Tpxq1bJy0zxOSZRP4bMNwW/OqMr/ttfTYkMbzU1eWrmMggl55Av1+YL/W2hal5mAE/j+tQzZY/z5cviaGnuTEOCKrRJDc81E212/JO/Mn0mXfJofz8q0jYqNnZXg0Xu52G7/jx40qlGlpxqKoYcNct89N3nzxwmwiMNmX2JjauQ9RuWfpt2WZ7DpVyZQ50GutDwF/KKVsXbobgF+BBcBw67HhgOtcwkrKlpdeJzaGewa+wL7o+nw4Zxzxlu0O5z4+K8PZHhIuudzG788/ye7SjbyVq3ik95N0PXKRwy+5bSXu6JkZVI0II8rk+OOhgGFdmpD+/M0S5IOA/TDc1notyGjQiiEb00Br/9dUEhWmXOmVwCPADGvGzS7gHowPj1lKqZHAXmBQOV8jJCXFmwsD5Ve922Aa0pvpM5/loaQx/HBR58LzCsowV27JzqHrhGUcyM6hYXQk4xuf5YonRlL1r6Pc2+85vm+RANaVtra2FM+0yc7JJdIUzrAuTVi+PavwuZIT4yTAB5GG0ZFFJta/uDSRCWlv0+nAdjaYL5bFU5VEuRZMaa0zrOPsl2qtk7TWx7TWf2mtb9Bat9Ja36i1PuqtxoaqW27uxKA7JrInpiEfznnRqE1SnrRXjGCvtabrD/O58u4kjp/JZ8Cwl40gb2Xfo7PtjmUvJzefRRsPFinrIEE+uBQfhlt48TWcrBJp9OqRyfTKQlbGBogqjcwMvmMCPzaPZ/zX7/L2/InUyvm71M+jMIZvYk8eZcrccby89C02NGxD7ztfc7pA60B2TpHdsYrLzsl1P8ErApZtmNC2ef2pqlEsuPga+mz7kdj8MzKZXkmUa8GUt1S2PHpnbEMnZ87l8sDP83hyxaecqFaD164exsxLbyI33LPysjXOnmbk2lTu/2Ue4bqAidcO55POfdDK+Wd6uFKFk76uKOD1wR2lNx/kbKud62zbxIJPR5Mx5j90/M8YfzdLlIPPF0x5kwR6Q2q6hSdmbSRfa9oc3s3Y7yZz5b7N/FmjNp93SCSt9ZUcatoalCrSAzfl55Kw/1duy1xNz03fUeNcDktaX8XE6+5mb0xDr7TNHB3JqpTuXnku4T+p6RYmLd3O5DfuxxQexq+LlpPUqZG/myXKSAJ9kGqesvj8wgOtuXb3Bkasm8+1uzcYx2JiOGpuypacCHRBAReePErzYweolneOsxEmFrW5mk869WFzg1ZebZesgA1+9hPuw9KXMP7rdxkw4k2GPdxf7taClKeBvrxZN8LLimRJKMUPF3Xmh4s6c2nYaRa0/Bt++YXamZm0P3SEg8fPYKl1IRlxCeRffTX/OdOQ01V8M7kmk3bBzz6nfn7ba3l6+Yf0W7+ESWmXSKAPcRLoA4yt4qB9BkykKZwR/a6CeDOMGAFAjPVPW+x6alXcL6oqC1kBGxrsUyn/rlqdxXFXc+u2FYw/fK8fWyUqgmTdBBhnWxK6W4HqbIPxshrWpQlvDO5YqtcXwaH4XdnMDjdR41wOvbavlBWyIU569AHIfjGVJ7y56GXuegsJTWvLxGsISk6MY/TMjMI5oHXmtuys3YhBm77hsbRb5cM8hEmPPgSUZvzcXUFkWRYfupLizUUrDCrFzEtv4jLLr0Rm/uavZokKIIE+BLgbP7cfhhnapYnbgmWyLD50Fd/3d94l3ckNC2fEb8v91CJREWToJgQkxZt5ceFWp6tbneW/JzStzaS0HS43F5EMm9BVfLL/SPUYvm91Bf23fAfnzkGVKn5uofAF6dGHiLF92jn01F1lyyTFm1mV0p03Bnf0+HtEaCheEgFgRvubqHrsL35+a6ofWyZ8SQJ9iChLtk5ZvkeEhlNn8wq/XtG8Ewdr1CHn/cmSfROiZOgmhJQ2W6es3yOC26S0HeTa1b8uCAtndvsbGbVmNv1nrSQpfrAfWyd8QXr0QlQyzibbZ196E+G6gKtXLfJDi4SvSaAXopJxNtn+R3R9VjbtwO1bvoWCAj+0SviSBHohKpnkxDhMYY4rKmZdejMNjx1i5eTZfmiV8CUJ9EJUMknxZiYN7FAk8wYgrfWVZFerwfH/vieTsiFGAr0QlVBSvJmMsTcXWUB1NqIKX7a7nht3rOJ/837xY+uEt0mgF6ISKz4xO+vSm6ian0eX1Uv91CLhC+UO9EqpcKVUulJqkfXfzZVSPyulMpVSM5VSstROiABVfGJ224UXsbF+K4Zu/aZcG9SLwOKNHv2jwDa7f08EXtdatwSOASO98BpCCB9IToxzWB39ZXwiLQ/tBtn1LWSUK9ArpRoBvYAp1n8roDswx3rKVCCpPK8hhPAd+9XRYGwWPzfuas6YqrJ74lt+bp3wlvL26N8AngJsibd1gGyttW199X5All0KEcCS4s2FPft8ra27T3Wl7qJ5LPzpd383T3hBmQO9Uqo3cFhrvb6M33+/UmqdUmpdVlZWWZshhPCC4ruUfXHpzdQ8e5qNr0/xY6uEt5SnR98VuFUptQf4AmPI5k0gWillq6HTCHCakKu1nqy1TtBaJ8TGxpajGUKI8iqefbO2UTt21jZz85rFfmqR8KYyB3qt9RitdSOtdTNgCLBMaz0UWA4MsJ42HJhf7lYKIXzKoSyCUsxufxOX798KO2THsWDnizz6fwKPK6UyMcbsP/TBawghvMhZ9s3i+JsoCA+Hjz7yU6uEt3ilTLHW+nvge+vXu4DLvfG8QoiKYStVPSltBweyc2gYHckTiR0J290bPvkExo8Hk6nkJxEBS+rRCyEAF3sT3HsvzJ8PixdDkmRKBysJ9EIIUtMtRXrzyYlxRtDv0QMaNIApUyTQBzGpdSNEJZeabmHMvM1YsnPQgCU7hzHzNhsVLCMi4J574KuvwCIVLYOVBHohKrniOfQAObn5PDFroxHsR4406t5MnuynForykkAvRCXnbGtBgHytGT0zg2aTt7Ey7grOvP0unD1bwa0T3iCBXohKztnWgja2+pXvdehNtaNHWD/xvYpplPAqCfRCVHLOcuiLW9W0A7/VaUKN/70j5YuDkAR6ISo5WwXLcOW4j2whpfgkoQ9xBzJh1aqKa5zwCgn0QgiS4s28OqhDiT37L9tez4nImvDmmxXYMuENEuiFEMD5nr0rOVWq8efAofDll7BvXwW2TJSXBHohRKGkeHORDcPtRUeaaDUuxRijf+edCm6ZKA8J9EKIIpxNzkaawnnh1nbQtCn07w/vvw/Hj/uphaK0JNALIYqw315QAeboSF7q1/58HZyUFDhxAt6TVMtgoXQApEolJCTodbIRsRDBIzERMjJgzx6IdJ2HL3xLKbVea53g7jzp0QshSm/MGDh8GD7+2N8tER6QQC+EKL1rr4UuXWDSJMjL83drhBtSplgI4VJquoUXFmwlOycXgJgoE2P7tDPG68eMgb59YeZMGDrUzy0VJZEevRDCqdR0C8mzNxYGeYBjp3NJnmOtatm7N1xyCfz735CfX8IzCX+TQC+EcGpS2g5yCxyTNXLzNZPSdkBYGLzwAmzbBp99VvENFB6TQC+EcMpV+eIij912G8THGwE/N9fl+cK/JNALIZwqqXxx4WNhYfCvf8GuXcYm4iIglTnQK6UaK6WWK6V+VUptVUo9aj1eWyn1jVLqd+vfMd5rrhCioiQnxmEKc17R8vo2sef/0bOnkYEzbhycOVNBrROlUZ4efR7whNa6LdAFGKWUagukAN9prVsB31n/LYQIMknxZiYN7ECkyTFMzF1vMSZkAZSC8eNh/35ZLRugvLYyVik1H3jb+uc6rfVBpVQD4HutdVxJ3ysrY4UIXF0nLMPiZLxeAdFRJrJP59KwVjXmLfgX9bZvhMxMqFOn4htaCVXoylilVDMgHvgZqKe1Pmh96BBQzxuvIYTwD1eTshoj3VIDluNnGNZ2EPnHT7Br1JMV2j7hXrkDvVKqBjAXeExrfcL+MW3cLji9ZVBK3a+UWqeUWpeVlVXeZgghfKSkSVl7v8c25bOOPWgy61O+m7vcx60SpVGuQK+UMmEE+Rla63nWw39ah2yw/n3Y2fdqrSdrrRO01gmxsbHOThFCBABP9pS1eb3bUE5XiaRqylM+bpUojfJk3SjgQ2Cb1vo1u4cWAMOtXw8H5pe9eUIIf/NoT1mro1G1ePOqIXTLXMf0MW8VHk9Nt9B1wjKapyym64Rl5ydyRYUo82SsUqob8COwGSiwHn4aY5x+FtAE2AsM0lofLem5ZDJWiMCXmm7hsZkZbs+LyM9j0SePcsHZU0z530Lm/XaiSBkFMDYyKVLjXpSJzydjtdYrtdZKa32p1rqj9c8SrfVfWusbtNattNY3ugvyQojgkBRvJibK5Pa8vPAInu7xMPX//ovGr/7HIcgD5OTmG2UURIWQlbFCCI+N7dMO9wM4sMF8MdM69WT4+kV0OOA8oJdUYsFGhny8QwK9EMJjSfFm52l0Tky6Zjh/1qjNK0veoFqu44pZd9k8qekWxszbjCU7x0jhzM5hzLzNEuzLQAK9EKJUzB6mW56sGkVyz8do9dcfjPm+6E5UpjDF6XN5JfbUJ6XtICe3aPnjsgz5yF2BbDwihCil5MQ4xszbXCQIR5rC6dSkFqt2Fp2SW9k8ng8T+jJy3XyWX5TA9y0uo0q4KlxsBUZPPXn2Rl5cuNVYZRsdSXJinMuhnZKGfFLTLUxK28GB7BwaRkdyfZtY5q63FLbVdlcAuJwILv4cyYlxQT9pLJuDCyFKzVkwnJS2w2mphKp550j99HHqnsqm991vcLhmXbfDP5GmcKqZwgo/DIpTnF+Jadv1CnD4ALI/z545OpJVKd2dXpezD7FAzRDyNOtGAr0Qwiuapyx2GcBbZe0lddoT7IhtypDbJ3Auwn32DoApXJGb7/0YpYDdE3o5HHdV18fVB4O/VWitGyGEKGly9ffYpjzRazSdDuxg3DfvgYcdzPwCTZST6pnlVSvS5HTc3tWwkLPg70ygzgfIGL0Qwiucjd3bWxrXlf9eOZhHVs9kX3R93r1ykNvnLNCQk1vg9rzSys7JJTsnl9qnj9Mm82csSz5kX3Qus7btJP/sOfLCwjlYM5bdtRuyusmlbG7QitR0S5HhG2/MB1QUCfRCCK+wBTNb8KsVaeJcXj6n7QL1a1cPpfHxQzy14lOORV7A5x17uH1ebw/cNM4+RJ9tK7j59zV0PPgbAPkqjMM169CiSSN+OwtRuWe4es8GBm75FoAjUbVYvLYXTJsIjRo5jOVbsnOYsWafQ1ttWUJJ8Wbnk7xrF8NllxnbMfqQjNELIXzKFuBswx8R+XlMnjee63at56mejzKn/Y2+b4TWdN27kbvXL+CGzLWEoclo0JpvWl7BL43bsbl+S86YqhETZSoyARydc4JuezK4ddsKbsj8hdzwCD64vB/vXjGAnCrVPH75ri1q89POo0U+COoUnGXdqwNRzz4LL75YpsuSyVghRECxn+islnuGyfP+zTV70hl//QimXN7PJ68Zee4M/bYuY/j6RbT+ax9Homoxo+MtfNEhkYMXlK5qbqNs407k1m0r2BXTkEdufYqt9VuWuW1X797AtFnPw9KlkJhYpueQQC+ECCip6RZGz8wo7NVWycvltUWv0nvHSma1v5Hnb3qQMybPe8klaZR9iLs2LGbwpq+pdfYUm+q35JPOfVjU5hqPM35cuXLvRl5b9Bp1Th9n7E0PejT85EzK8o8YsW4BVY4fgxo1yvQcEuiFEAHn2dTNRcaywwryeXTVFzz60+dsr9uUp3s8zAbzxR49V/EceaUL6LYng7s2LOaGzF8oUIqv4rrycedb2WBuY+xt6yXROSd4Y+GrXLd7Pe9dMYCXr70LrUqXHbTk40c4U+MCOu10XxHUFQn0QoiAlJpu4YlZG8m3iz3X7lrPhK/eosHJv5h7SXfeu2IAmXWbuHyOSFM4/TubWb49C7V3D7fvWcOQjKXUOfQHR6Jq8UWHRKZ37MmhC+oCRj7+4MsaM33NPq9dR3hBPuO+eY+hGUuZf/G1PNnrMXLDPbtbiD15jLXv3MnWh1No99+XytwGTwO9ZN0IISqULTvHPmvlh4s60/v/JvPIT18w+OcF9N+yjJ8bX8K3LS4no2Fr9sSYOW2qShiaS9RpHrswnytWL4fvvoP1640n7tqVsdfexeeNEhyGZ6pXiWB8UnuWb8/yOCfenfywcJ65eRSWCy7kqRWfUuvMSf4vaYxHk7Q9d6wEYEJ4C1amLPZ5qQUJ9EKIClc8FdMIdB3h9ivoPr0//dYupteOlTzz/Ueun8RkgoQEePll6N8fLrqIT12szj1urYnvLte/1JTi3SsH8VdULf6T9g7TZz7LiAFjOR5Z0+W3mKMjuW/PKrbVu4gfqzUAfJ9zL0M3QoiAYj+0U+/vI8Rl7aXZsQNUzcvlgigTj9xxNVx0kZF7Xq1o79lVCYPoSBPVq0YU5vcrBdmnc72ao5+44yfeWvgye2IactegcfxZs67DOeboSFbdHA2dOzvNNiptqQUpgSCECEpJ8WZeHdSBSFM4f9asy4qLOvNp5z5M7zaQxuOfg6FD4corHYI8ON/I3BSmOHUur7CufXZOLmdyC3h9cEeXJZfN0ZEel2O2SYu7irsHjqPhiSzmTn+K5keLlj+INIVzfZtYVg57mONVqzOzg2NKpSebsZSFBHohRMCxbUhujo5EYQReTypIOvu+GtUiHAqj2VasOvtgiDSFk5wY5/Sx4hTwxuCOREcacwKrm17KkNtfolreWWbPeIpLDmUWec2jn3xGt22r+d8V/fm7anWH53O3GUtZydCNECKkuaqqaatgWVL9+eKreouzDbUUHzJqftTCtJnPEXvqGNPje/JD8060ydrDEz9O59cLL2LQ0AkOGTplKYcsWTdCCIHRS3YWqG2956R4s8vganvMVZ365MQ4wHHIZXdtM7fd+SpP/TCVu9cvZOS6+QD80LwTj/Z50iHI22rqB13WjVKqB/AmEA5M0VpP8NVrCSGEK652xLIFaU84zxI63/N39mGSVSOG5F6P8e/uI4jL2ktW9Rh21Wnk9PnP+KBCpz2fDN0opcKB34CbgP3AWuB2rfWvzs6XoRshhC/5entAVz3+knbJKq4sm5v4e+jmciBTa73L2pgvgL6A00AvhBC+VNLwjLeeHxx7/OC4vaErvsq4Ad8FejPwh92/9wNX+Oi1hBDC70r6MLH/ADh9Ls9pL99XGTfgx8lYpdT9wP0ATZq4rmkhhBDBrPgHgLuJXV/wVR69BWhs9+9G1mOFtNaTtdYJWuuE2NjS1YUWQohgVdY1AuXhqx79WqCVUqo5RoAfAtzho9cSQoig4us5g+J8Eui11nlKqYeBNIz0yo+01lt98VpCCCFK5rMxeq31EmCJr55fCCGEZ6TWjRBChDgJ9EIIEeIk0AshRIgLiOqVSqksYG8ZvrUucMTLzQl0cs2VR2W8brnm0mmqtXabnx4Qgb6slFLrPKnzEErkmiuPynjdcs2+IUM3QggR4iTQCyFEiAv2QD/Z3w3wA7nmyqMyXrdcsw8E9Ri9EEII94K9Ry+EEMKNoA30SqkeSqkdSqlMpVSKv9vjK0qpPUqpzUqpDKXUOuux2kqpb5RSv1v/jvF3O8tDKfWRUuqwUmqL3TGn16gMb1nf901KqU7+a3nZubjmF5RSFut7naGU6mn32BjrNe9QSiX6p9Xlo5RqrJRarpT6VSm1VSn1qPV4yL7XJVxzxb7XWuug+4NRKG0ncBFQBdgItPV3u3x0rXuAusWOvQykWL9OASb6u53lvMZrgE7AFnfXCPQEvgIU0AX42d/t9+I1vwA86eTcttaf8apAc+vPfri/r6EM19wA6GT9uibGdqNtQ/m9LuGaK/S9DtYefeFWhVrrc4Btq8LKoi8w1fr1VCDJj20pN631CuBoscOurrEv8Kk2rAGilVINKqal3uPiml3pC3yhtT6rtd4NZGL8DgQVrfVBrfUG69d/A9swdqML2fe6hGt2xSfvdbAGemdbFVZcceeKpYGvlVLrrbtyAdTTWh+0fn0IqOefpvmUq2sM9ff+YeswxUd2Q3Ihd81KqWZAPPAzleS9LnbNUIHvdbAG+sqkm9a6E3ALMEopdY39g9q43wvp1KnKcI1W7wEtgI7AQeBV/zbHN5RSNYC5wGNa6xP2j4Xqe+3kmiv0vQ7WQO92q8JQobW2WP8+DHyJcRv3p+0W1vr3Yf+10GdcXWPIvvda6z+11vla6wLgA87fsofMNSulTBgBb4bWep71cEi/186uuaLf62AN9IVbFSqlqmBsVbjAz23yOqVUdaVUTdvXwM3AFoxrHW49bTgw3z8t9ClX17gAuMuakdEFOG532x/Uio0/34bxXoNxzUOUUlWt23O2An6p6PaVl1JKAR8C27TWr9k9FLLvtatrrvD32t+z0uWYze6JMYO9E3jG3+3x0TVehDEDvxHYartOoA7wHfA78C1Q299tLed1fo5x+5qLMSY50tU1YmRgvGN93zcDCf5uvxeveZr1mjZZf+Eb2J3/jPWadwC3+Lv9ZbzmbhjDMpuADOufnqH8XpdwzRX6XsvKWCGECHHBOnQjhBDCQxLohRAixEmgF0KIECeBXgghQpwEeiGECHES6IUQIsRJoBdCiBAngV4IIULc/wNUJ59FvXX/iAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get model predictions for the trainint set.\n",
    "predictions_num = 1000\n",
    "x_predictions = np.linspace(x.min(), x.max(), predictions_num).reshape(predictions_num, 1);\n",
    "y_predictions = linear_regression.predict(x_predictions)\n",
    "\n",
    "# Plot training data with predictions.\n",
    "plt.scatter(x, y, label='Training Dataset')\n",
    "plt.plot(x_predictions, y_predictions, 'r', label='Prediction')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may see from the plot how well our model predicts the training set examples."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
